The Pragmatic Engineer
01:04:28 · Gergely Orosz, Titus Winters
[01:04:28] S0: a It makes a difference for [01:04:31] S1: many metrics. Yes. Right. You know? And so like all of those things, like that is the form of impact. Then there's engineering productivity, or we phrased it as engineering capacity. Because if we save you, you know, 5% productivity gains, capacity gains, you don't get to spend that somewhere else. You just have more capacity on doing more engineering. [01:04:52] S0: Or having slack time for innovation or picking up when something happens, that kind of stuff.
01:04:52 · Gergely Orosz, Titus Winters
[01:04:52] S0: Or having slack time for innovation or picking up when something happens, that kind of stuff. [01:04:58] S1: Right. But they're not like dollar fungible. Exactly. You don't get money in a different account. And then the fourth form of impact, and this ties back to the AI thing. Fourth form of impact is changing the game. Right? Strategic innovation. Like, research makes impossible things possible. Oh, yeah. Security makes incredibly bad things impossible. Right? But if you change the nature of what we are doing, the cost structure, etcetera, then that's a different game. And the problem that I think that we see with AI, even with a clever you know, index metric approach to trying to capture the gestalt of productivity, if you're not looking for the value of a given change is substantially different, then there's no way you'll ever be able to capture that in the signal. Right? And we don't have anything that says the value of a change of this size has changed. So for instance, it is entirely possible at this point, and I hang out in security spaces and do a lot of low level c plus plus library stuff, you can write a JPEG parser in a couple weeks. Yep. Right? With whatever feat... Additional features you need. That would be insane to say that word... That sentence out loud three years ago.
01:06:18 · Gergely Orosz
[01:06:18] S0: Okay. So so what what what you're saying is what none of the measurements really capture is the possibility, the impact, the the effort it is to do a massive innovation, which right now with AI is happening and Right. And I'll I'll I'll give you also one example that I... And tell me if if this is what you mean with it is. There's a friend who works at a CTO at an engineering company about like a midsize engineering company. They're they're a niche industry. And, you know, like the the CTO, they they have been looking at like release frequency of the usual metrics, talk with their engineering team, are they happy, are they not? And of course, now they're adopting AI as well. They're seeing that, well, people are more efficient and just privately... See, this person was telling me, like, look, like, I kind of know that for a fact that we could probably do the same amount of work with, like, half the amount of people and then we could be cash flow positive. But, of course, there's a there's a morale thing. So he's, like, also, like, he likes the people. So it's like, I I don't... I'm not sure I want the emotional package on on me, but we can see some people are picking it up faster and some slower, and we kinda left it at that. A few weeks later, I'm talking with this person and says like, you wouldn't believe, but we're probably gonna be cash flow positive. I'm like, how? Like, did you go ahead with the cut? He's like, no. What happened is one of our engineers came to me and said like, we license this component, think of a library, like it's a binary Yeah. But that was written by a vendor in the space that has a high performance thing that they use in their in their software. It's it's a core part of it. And they have a revenue share where they they pay like, I don't know, like five, 10, 15% of their revenue for this component. It's so important. It's obviously licensed, closed source, but it's it's really important and it would take them years of a five person or 10 person team to write it and it might or might not work. So it's not worth it. Then Gergely came and said, like, what if what if we just gave the specs? Because they have all massive amounts of tests around this thing, a test harness. They gave it to AI. AI wrote a component to them which is faster and better what they have. No prior domain knowledge.
01:08:21 · Gergely Orosz
And at first they thought, well, maybe we'll open source it, but then they told the supplier and the supplier said, like, it's a huge business for them. They're paying them to To not. To not open source it and to give them the implementation, which they can relicense. And suddenly Yeah. The game was completely changed. Like like, if if this person would have gone ahead with, let's say, the layoffs, this engineer might have just got... And no one might have known that the game could be changed Yeah. Because of this AI makes things crazy things sometimes possible. Right. Exactly.
01:08:51 · Titus Winters
[01:08:51] S1: And so I think that this is the insight that tries to bridge the gap between... I have not seen a research based number that says productivity post AI is up more than like 20 or 30%, But then everyone talks about, like, these 10 x sort of experiences. I super don't think it's actually 10 x. But... Yeah. But, like, how do you bridge that gap? And I think it's exactly this thing of a lot of the value of AI is not in doing the same thing faster. It's doing a harder, more valuable thing that you wouldn't have even thought about. And if your productivity measures are the logical equivalent of counting how many widgets are coming out of the factory, and then you start producing super gold plated deluxe widgets, there's no way to measure that or track that over time because we just don't have the tracking of what's the value of this component or feature. And that also matches with, you know, talking with senior engineers that are like, yes. We're up 70% on amount of code. We're up 30% on outages. Number of features is the same. The thing that we can't know for sure is, are the value of those features up? Because what's... How do you how do you do that?
01:10:08 · Gergely Orosz
[01:10:08] S0: One challenge that AI is is giving is entering the market for for new graduates. There's now... There seems to be expectation. Companies are very eager to hire senior engineers who know how to use AI, and they have taste and judgment and we have these things thrown around. It's not deliberate, but whenever I talk to the AI labs, they always tell me increasingly that they look for taste, judgment, agency, these Let's kind of go back to to education, to to computer science education. What are your views of how today's computer science education works, especially for how we know the industry is changing? And what are some of your ideas of what could possibly be changed?