00:49:20 · S5, Max Read
[00:49:20] S5: This message is supported by ADT. It's all back. Back to school, back to work, back to the to do lists. ADT wants to help you get back to home. Their security systems protect you and yours so you can appreciate every moment. With systems that can be installed by pros and are accessible right from the ADT plus app. So you're always in control when you wanna be. With cameras, sensors, and more doing what they do best, keeping an eye out. And if you ever need help, ADT's reps are on call 20 four seven. When every second counts, count on ADT. Visit adt.com today. [00:49:57] S1: Welcome back. I'm with Erin, Jasmine, and Mike, and we're gonna be talking about stories that we're tracking this week. Mike, what story are you tracking this week?
00:49:57 · Max Read, Mike Isaac
[00:49:57] S1: Welcome back. I'm with Erin, Jasmine, and Mike, and we're gonna be talking about stories that we're tracking this week. Mike, what story are you tracking this week? [00:50:05] S4: So I... This is kinda nerdy, but that's my thing. I found this memo from the VP and distinguished engineer at Amazon this week that, you know, most normal people are not tracking. But I think it's really interesting because it's essentially a very long manifesto on the use of AI among software engineers inside of the big companies. And what they essentially cop to is token maxing is kind of a a terrible metric to measure how you're doing as an engineer, As is LOC, lines of code, the amount of line... Of of code you are contributing building these products, also a messy metric, which basically means you're larding up these systems with a bunch of stuff that someone's gonna eventually have to delete. And so in this long memo, they essentially tell employees, look. What we care about are long term outcomes. This is, like, what really matters in the end, and this is what, you know, we need to be judged by. The problem is you can't really judge long term outcomes on a quarterly basis when you're trying to figure out who should still be employed here. So I think after, you know, thousands of words, we end up kind of back where we started, and I'm very curious how they judge these engineers in the coming quarters.
00:51:18 · Erin Griffith, Mike Isaac, Max Read
[00:51:18] S3: I mean, that's so interesting because I feel like when the token maxing trend started, everyone with any common sense was just kinda like, this seems absurd. Yeah. Like, just spending as much money as possible to to write as much code as possible. Like, the... Playing the volume game just... Obviously, that's gonna end badly. And we call them all new as a short term bump. And the token minning or whatever they call it, token minning. The the reverse of that trend happened so fast. Like, by the time we could barely get our first story up, it was already reversed. But, like, it seems like they're they're still struggling to figure out where the balance is. Is that, like, kind of your upshot from this? [00:51:57] S4: Well, what better metaphor for the tech industry than spending billions of dollars to find out something is a terrible idea and ending up back where you started. [00:52:06] S3: They're in the arena trying things, Mike. [00:52:08] S4: Yeah. Exactly. That's true. [00:52:09] S1: I am This is... I have to say this... You say this is nerdy, but this feels like the most relatable possible thing to me. Everybody has a job where they introduce a new metric, and you know the metric's stupid, and you fulfill the metric, and then your bosses are like, actually, we're gonna do the opposite metric. And so you do that, and you're like, this is still stupid. And then they're like, they're then sort of like, actually, metric is, the long term health of the company. And you're like, well, I'm still just gonna do my job and let [00:52:33] S3: me You're self employed.
00:52:33 · Erin Griffith, Mike Isaac, Max Read, Jasmine Sun
[00:52:33] S3: me You're self employed. [00:52:34] S4: What are you talking about? Like, right. You are your own boss. [00:52:37] S1: Well, I am also self hating, so I think I can. Jasmine, what are you tracking this week? [00:52:43] S2: Oh, man. I saw that this this new research report that a team at Anthropic put out, they did a randomized controlled trial trying to understand how much experts defer to AI systems. So they basically gave a bunch of professional, like, statistical analysts access to AIs to give them advice on a decision in a scenario. And what they found was that not only did the experts, I think over 80% of the time, make the same decision as the AI, which are split half half, they also, over 70% of the time, use the AI's arguments to justify their decision, but only 15% of the time did they say, oh, yeah. I was influenced by this AI agent. And so you... We have this world, which I think is one that we're gonna be in increasingly, where humans are oftentimes almost becoming, like, rappers on AI agents and AI systems where, like, all we're doing is, like, seeing the faces and, like, typing the thoughts. And, like, I feel like what, like, there's all this concern about AI takeover and rogue AIs. And I think that a lot of the times, we're just gonna let the AIs take over because we'll just be outsourcing all our decisions and we won't even realize. So that's what I'm worried about.
00:52:43 · Jasmine Sun, Max Read
[00:52:43] S2: Oh, man. I saw that this this new research report that a team at Anthropic put out, they did a randomized controlled trial trying to understand how much experts defer to AI systems. So they basically gave a bunch of professional, like, statistical analysts access to AIs to give them advice on a decision in a scenario. And what they found was that not only did the experts, I think over 80% of the time, make the same decision as the AI, which are split half half, they also, over 70% of the time, use the AI's arguments to justify their decision, but only 15% of the time did they say, oh, yeah. I was influenced by this AI agent. And so you... We have this world, which I think is one that we're gonna be in increasingly, where humans are oftentimes almost becoming, like, rappers on AI agents and AI systems where, like, all we're doing is, like, seeing the faces and, like, typing the thoughts. And, like, I feel like what, like, there's all this concern about AI takeover and rogue AIs. And I think that a lot of the times, we're just gonna let the AIs take over because we'll just be outsourcing all our decisions and we won't even realize. So that's what I'm worried about. [00:53:49] S1: That is a cheery thought. I mean, as somebody who professionally takes other people's opinions and repeats them as though they were my own, like, I I do I do kind of sympathize. It's just usually the people I'm stealing from are people, not, you know, not artificial intelligence. [00:54:01] S2: People say that, like, AI makes everybody a manager. Like, here we are.
00:54:01 · Jasmine Sun, Max Read
[00:54:01] S2: People say that, like, AI makes everybody a manager. Like, here we are. [00:54:05] S1: Yeah. Exactly. That's exactly... This this is exactly what managers do. Wow. Erin, what about you? What are you tracking?