00:10:57 · David Flickinger
[00:10:57] S1: Sure. I think this is a a quintessential example of an agentic AI architecture in a business. And so the example is a roofing business that is constantly bidding on commercial projects. And so they do a lot of new construction, which if you think about it is a bit like a math problem because the building doesn't exist yet except on plans and in specification documents. And so what a good estimator does is they will read those documents. They'll use their homegrown spreadsheets and whatever tools they have. Maybe they're using some third party software tools. But, ultimately, they're putting their own logic in, their own experience, their own materials pricing, their own assumptions about labor, and they're putting forth a proposal based on their best guess or estimate. And it was very clear in this particular business, we could use an AI orchestrator to task AI sub agents to do similar things. And so in this example, we have an orchestrator agent that tasks a plan reader agent that goes and reads these architectural plans, which are quite different actually from a basic PDF with just words. These have numbers. These have measurements. So you need to almost go pixel by pixel to make sure that you've extracted every number correctly because you pick a number wrong out of a spec document, and it could be wildly different outcome in your proposal. So that's one work stream. Second is, okay. Now that we've identified the type of building, can we have an agent go reference all similar projects that this company has done that are like this? Hey. This is an apartment complex building with five large HVAC units on the roof. How many similar buildings have we done like this with this sort of design? Let me go reference those and understand what those are. Another agent is going pulling the latest materials pricing. Another agent is pulling the latest labor pricing. Another agent is trying to understand risks in the job, whether that's geographical risks because it's a 100 miles from the headquarters, and so there's travel, or we need cranes to lift, materials that are unusual because of the dimensions of the building or how close it is to other things. So you have all these agents that are doing tasks. They're routing this up to the orchestrator agent.
00:13:32 · David Flickinger
And, ultimately, that orchestrator agent is applying the rules and logic of the business to generate a proposal. That's really what the estimator is doing, what the human's doing. He's doing all that in his head. And so as we started to iterate this in the business, this example I'm about to give you was sort of the ultimate test of, hey. Does this work? So the estimator, this... Roofing business had really big job. 47 individual buildings in this very large complex in a large metro area in South Florida. And the estimator spent a couple weeks not using AI. Just, hey. This is too big of a project. They're gonna do it on their own, and this AI may not be ready for prime time. And, indeed, we hadn't actually deployed it live where it was being used every day. We were still iterating a bit, but we were pretty sure that we had it right. So we said, well, we'll take all the documents. We'll have it. We're not gonna interact. It'll be a blind test. And so we ran our simulation with the AI, and it took about twenty three minutes to go through this massive pool of documents, 47 plans, specification sheets, and generate a number. It took the human estimator about two weeks to generate a number. And on a $6,500,000 estimate, they were within $400 of each other. And even us working and having designed this system, we were pretty blown away, but it was just proof and validation that this architecture works when the agent has access to the same logic that the business owner or key person in the business has. It can make nearly the same decision, and and why shouldn't it? And so that was the moment, and there's a great story about that that roofing business in the sense that they have leveraged this architecture to grow tremendously over the last several months without having to hire additional people. Now they are replacing people with AI. They're making their estimators more valuable and productive through the supplementation use of these tools and these architectures in their business. But it's a really fun example because the size of the project was so large and the outcome was so close. Twenty three minutes for the AI, two out...
00:15:58 · David Flickinger, Mikk
Two weeks for the human estimator arriving at the same number, and just the cost savings and productivity gain when you think about that applied across a year's worth of work is really, really cool to think about and, is a lot of fun to be on the implementation side and see these success stories. [00:16:18] S0: My question was and still is, we're talking about the senior estimator who has decades of experience doing this particular thing. Can you explain the logic behind the AI? How they're able to learn this all this experience in such a short period of time?
00:16:39 · David Flickinger
[00:16:39] S1: That is really the the key question. And I think in order to answer that question, you have to understand a little bit about the architecture. And so what the AI will do is it will start to build this memory and context layer, which becomes its foundation. So it uses access to all the documents. In this instance, there were over 3,700 proposals that were used to fill this initial memory bank of estimating logic. And so the AI doesn't get tired, doesn't get bored. It goes and extracts all the logic out of 3,700 spreadsheets and distills that out into rules. Similarly, it pulls the language of proposals and extracts that out. It looks at jobs that were lost and jobs that were won, and it looks at not only one jobs, but the job costing after the fact. Hey. We estimated a 16% margin on this job, but it turned out it was 12%. Why? Well, the labor was underestimated. So it can fix that in a subsequent iteration. And so it builds this layer of contextual memories as well as rules, logic gates, things like this that are really the human decision making points that have been distilled out into technological gates or rules. And what you try to do is closely mimic the way that the human makes a decision. And, of course, not every decision that a human makes can be distilled out into an AI rule book. But it turns out that a large majority of these reproducible tasks, whether it's creating proposals or responding to customer inquiries or billing and collections, can be written into rules. And as you start to think about this, and maybe this is something we'll get into later in the in the podcast, is how that knowledge, which we call tribal knowledge or typically lives in the minds of the key people of the business. If you distill that into an architecture that lives and persists and that AI agents can interact with, does that make the business more valuable? And, Mikk, without doing a deep dive into software and coding and databases, what's happening is an AI agent is deploying swarms of sub agents to go look into all these areas of the business that it has knowledge about. And that agent, that orchestrator agent can only give you an answer as good as the systems it has access to. So the answer is if you hadn't connected it to anything, it couldn't give you the same answer as the chief estimator because it would just use its generic logic.
00:19:36 · David Flickinger, Mikk
But if you had connected it into the systems, into the CRMs with thousands and thousands of customer records, into the supplier portals that has up to date pricing, into your labor and payroll data. So you know if you're putting a seven man crew on this job, you know what that's going to cost with a high degree of predictability. It can give you a very good answer. And so in short, it can give you these answers if it's connected into the systems. And that is where we really spend the majority of our focus in an engagement is first and foremost connecting an agent into your systems so that it has the context so that it can start to iterate and learn. [00:20:26] S0: So as a business owner operator, from which moment will you be sure about this this work the AI was just doing when it comes to this particular offer? From which moment are you so sure that you are willing to send this to a client and making sure that it's it's all correct? Because people make mistakes. I guess AI is making mistakes. And I've been flying recently quite a lot, and, like, I I like to read, like, planes and things, and they have those pilots double checking, triple check... Checking things like this. How do you make sure that this... What you just sent to a customer is is correct and you can you can actually sleep well on it?
00:21:05 · David Flickinger
[00:21:05] S1: So we design a workflow that has human checkpoints at key instances along the way. And that system, the goal would be that you, over time, remove those human checkpoints as you validate them. So you start with a lot of human checkpoints is the answer. You don't wanna have AI going and emailing proposals to customers without you having checked it out, of course. And you wanna design the system such that you feel very confident at each step along the way that you are either approving the next step, the next phase, or you've approved it enough times, you're confident that you only need to approve it here. But even in those instances where you say, hey. Full approval. Actually, go email the customer. Do the proposal and email it. Let's say you're there. We don't have any clients that are just doing that yet, by the way. We do have clients who are reviewing a final proposal in their draft emails, clicking the PDF and saying, well, that looks good. Send. So, of course, they could. But what it will do is identify anomalies and red flags as well. So if there's a confidence interval below a certain threshold, it will say, this is ready for your review, but please let me point you to here, here, and here because this is worthy of additional investigation. This is an anomaly. I haven't seen this before. And so the goal is not to take the human out of the loop for these critical things. The goal is to make them more efficient and hopefully direct their attention to things humans are best at. I think ultimately that's one of the promises of AI more broadly stepping out and zooming out of what we done is directing the human's attention to things we are best at and letting AI do things it is best at, whether that's repetitive tasks, data analysis, data extraction. I got a 30 page contract yesterday. And even if I was an attorney and, of course, forward that to my attorney, I wouldn't even really know where to look to sit down and read a 30 page document. But this is an example anyone can do is you can put that document into ChatGPT and say, hey. Where are the most critical areas I should look? What are the red flags? What seems unusual in this contract? It can direct you.