The Pragmatic Engineer
00:30:32 · Titus Winters, Gergely Orosz
[00:30:32] S1: Think if you're looking for it, it's already out there. In the Java space, particular, tools like OpenRewrite, like, you basically... There's a little configuration per run for it, and you say, like, this is the pattern I'm looking for in the Java code. This is how the rewrite would look. And it goes through and blasts through all your Java source files and and does that update. Turns out, AI tools are really, really good at generating open rewrite recipes, which is way cheaper on token consumption, and way more accurate, and much faster, and auditable. Right? It is a substantially better pattern. And I... We've seen talks from, I wanna say, Netflix and Uber. I could be wrong. Specifically on, like, everyone seems to be coming to the same conclusion of what you need is a deterministic tool that declares the intent of the rewrite. You can have the AI, like, write a little skill to tell... Train the AI how to create the config for those tools, and then you move on with life. In the c plus plus space, actually, some good friends might have a startup right now. It's called BrontoSource that is specifically just about that sort of declarative rewrite pattern at a very high level of intent, very powerful for C plus plus Yeah. [00:31:50] S0: I I like the idea of of using AI to help generate deterministic software tools transformations.
00:31:50 · Gergely Orosz, Titus Winters
[00:31:50] S0: I I like the idea of of using AI to help generate deterministic software tools transformations. [00:31:58] S1: Yeah. Kelsey Hightower gave a great talk recently on, like, zero token architecture. And, like, his whole thesis, and I think everyone is coming to the same conclusion, don't use AI for constant inference. Use AI to generate the deterministic tool that will actually do the work, and then have AI fill in the gaps on the prob... Parts of the problem that you can't solve or where to invoke the tool, things like that. But, like, if your whole architecture is token consumption for all time, it's probably a cheaper and more effective way to do it.
00:32:35 · Gergely Orosz
[00:32:35] S0: Yeah. I just recently heard a head of engineering tell me that they're using these AI code review tools, which which are pretty good, but very expensive and slow. Right. And this person said, hang on. Like, what if we told the AI, okay. Like, let's do a thorough code review every now and then, but wanna find something. Let's have it generate a lint rule. Yes, please. Which is cheap and repeatable and it will no longer take thirty minutes the the cost. And also, it's it's just nondeterministic. It might or might not find it next time. That that that was frustrating. Like, this person was like, look, when we know we have certain things, so now what what they're doing is every now and then they still run some of these reviews and of course... But whenever they find any issues, then they fire up inference AI, several agents and say, alright. Can you come up with an improvement in our deterministic linting Yep. Can help it. And and the inspiration came actually from this person heard about how Uber... Actually, I I think it was... Heard heard me talk about when I joined Uber, I could barely commit my code the first first week. It was so painful because the linter just kept rejecting it because Uber had this Right. Enduring culture that Yep. Pre AI, whenever you saw something break, you just put a lint rule. Architectural rules were all in runt rules. You could not have something like a... We we had an architecture and and you could not inherit the wrong way because a linture would say like, uh-uh. You cannot you cannot inherit from, like, a builder cannot inherit from a... I don't know what what a composer or whatever that was.
00:34:03 · Titus Winters
[00:34:03] S1: Yeah. No. I I think it makes perfect sense. Like, Let's try to learn our hard lessons only once, codify those rules somewhere deterministic, apply them equally and appropriately and fairly, and leave the interesting parts of code review. Make sure that the parts of code review that we're focusing on are only the things that humans need to be doing. Right? Which was true then. I don't think everyone invested in linting and static analysis that quite to that extent they should have. And then it's equally true now. Right? Like, we can't review everything, but, like, there are places that require... Like, this human attention is is well spent.
00:34:45 · Gergely Orosz
[00:34:45] S0: Titus just made the case for learning the hard lessons once, then writing them as rules, and saving the human attention for the things that really need such. That is pretty much the same way you want permissions for AI agents to work, which brings us to our seasonal sponsor WorkOS. WorkOS provides the auth infrastructure that b to b and AI native companies need to sell to enterprise customers with SSO SCIM RBAC and now authorization for AI agents. WorkOS Airlock is intent based access control. Most access control only asks who you are. Airlock also looks at what the agent is trying to do at the moment it tries to do it. Clear cut rules like no agent deletes cloud resources are run as plain code. For the fuzzier calls, an AI just evaluates the policy at runtime. You write those policies in plain English and Claude can pull the ones you already have out of Notion or Jira and import them for you. Your agents, whether that's Claude or Devin, connect to a single access point, and your MCP servers are behind this access point. Airlock is a new layer on top of your existing permissions which means that an agent can never get more access than the person who invoked the agent. When something risky comes up Airlock can escalate to a human for approval, say over Slack. It's the same idea Titus described: Write the rules once and spend human attention only on the calls that need it. You can request early access at workhorse.com/airlock. I also want to mention our presenting sponsor turbopuffer, a ridiculously scalable, fast and cheap hybrid search engine built on top of object storage. How often do you get to see a rewrite playing out in real time? With the TPUF team, we finally can. They are completely redesigning their storage architecture from first principles to make search faster, cheaper, and more reliable at scale. The new system called TurboPuffer v three is not yet in prod, but it already passes CI. And when the rewrite started in September, it was not nearly as performant as the current production version v two. The tfifth team had been perf maxing query plans until they are fast or faster than v two, doing all this in public all the way up to the v three production launch. And it's so cool to see their progress.
00:36:41 · Gergely Orosz
The new system went from being 10 to 100 times slower across most metrics to being several times faster on things like cold vector search, order by attitude, or hybrid search. The v three system is still slower in areas like full text search, but only one to two times slower versus a 100 times slower when this perf maxing started. And the t puff team is sharing exactly how they optimize things in their development logs with things like group count aggregation, compressed column encodings, batch scoring, and many more. A lot of these wins are basic foundational work right now, but the team will be sharing some very cool systems work over the next few weeks. But hang on, why is the team doing a full rewrite? It's because search is changing and so is turbopuffer. Tpuff started as a vector database but it is no longer a vector database. It has evolved into a high performance database for search. The V3 discards a vector primary index and all indexes will share the same columnar storage system. Every query will run on vectorized query engines which is something you don't usually see in search databases but which will result in both very fast search and just generally a very fast database. The TurboPuffer team is keeping a work log of their journey and you can follow along at turbopuffer.com/v3. It's a blast to see how it's being built. Speaking of hard lessons and documenting them, we talk about Software Engineering at Google, the book that you're quite well known for? How did the idea come? How did it come together? What was your motivation to start with? So this