Plant Floor to Cloud: The Podcast, Episode 1 - Prove It Before You Scale It

0:02 I'm Tony Gondek, and this is Plant Floor to Cloud, the podcast. The mic's sometimes in shot, and I'll occasionally read off of a script. If the vendor pitching you AI won't show you the seams, I will at least show you mine. Quick context if this is your first time here. This is the monthly companion to the Plant Floor to Cloud newsletter. Same charter as every Tuesday issue, mid-market manufacturing, practitioner-level detail, zero vendor hype. If you ever hear me say, "Unlock synergies," on this show, something has gone very wrong, and I'd appreciate a reply letting me know.

0:44 The newsletter tells you what happened each week. The podcast is where I'm pulling one thread all the way through four of last month's stories, instead of skimming all four. We'll go about twenty-five minutes. Here's where we start. [explosion] [gentle music] There's something I hear almost every week in almost the same exact words. "I feel like everyone else has this AI thing figured out, and I don't even know where to get started." If that's you, I want to tell you something before we get into anything else.

1:37 Most of the people who look like they've got it figured out are only three weeks ahead of you, not three years. A fair number of them are currently staring at a dashboard that says everything is fine, when in actuality it isn't. We're going to look at exactly what that looks like in this episode because here's the thing nobody tells you when you're mystified by AI, the confusion usually isn't about AI at all. It's about a question that sounds simple and isn't. Where does this thing actually go in a job I already understand?

2:14 Get that question right, and the technology part gets almost boring. Get it wrong, and it doesn't matter how good the model is. This episode has one theme, one seam pulled all the way through, the difference between a demo and a job. They get confused constantly, and that one mix-up is behind nearly everything that went sideways in Plant Floor to Cloud this past July. So let's get into it. But before I share this story, because I know some of you are here specifically because you feel behind, two questions to get started.

2:57 Not for tests, not a framework, two questions you can ask about literally anything before you spend a single dollar or a single minute on AI. Question number one, where's the friction already? Not what could AI theoretically do somewhere. Where in a job you already understand cold does something visibly and specifically slow down or go wrong today? Is it a form that takes too long to fill out? A question customers ask forty times a day? An order that gets keyed into three systems by three different people?

3:41 That's the entire search space. If you can't point at the friction, you're not ready to point AI at anything at all. You're shopping for a solution to a problem you haven't found yet, and that's how you end up with a chatbot nobody asked for. Question number two, if you fix that friction, would it actually move a number somebody cares about? Not, "It'd be nice." Would the solution move a number that actually shows up in a budget conversation? Yes to both, you've got a legitimate starting point. Can't answer either one, you're not behind, you just haven't found the job yet, which is a very different problem than the one everyone's mystified about, and it's the one we're solving in this discussion.

4:35 So here's the story from past weeks that I keep coming back to because it's really the cleanest example I've got of a demo and a job getting mixed up with real money attached. A mid-market manufacturer had a reordering agent running quietly for about six weeks. When stock on a part dropped below the reorder threshold point, the agent would draft a purchase order, check it against the preferred supplier list and the current price file, and then hand the results to a buyer to click Approve. It was good.

5:12 It was right almost every single time. So somebody asked the question that always eventually gets asked. If it's right almost every time, why is a person still sitting there clicking Approve? They took the person out of the loop on a Friday because nothing ever goes wrong on a Friday, right? Then over the weekend, an upstream price file came in with malformed data, a decimal in the wrong place. It's the kind of thing that happens also on a Tuesday, except on a Tuesday, a human catches it in about four seconds.

5:54 After that, every order the agent placed that weekend was individually plausible. Nothing looked obviously wrong line by line. Collectively, by Monday morning, everything was a mess. Nobody had actually approved of, because the entire point of Friday's change was that nobody had to. Uh, nothing ever happens on a Friday became a seriously freaky Friday because something did happen. Now, here's the part that should really bother you more than the dollar figure did. The model did not make a mistake.

6:37 It did exactly what it was built to do, exactly as well as it had been doing for six weeks. The agent's job, read the price file, check the supplier list, draft the order. It did that job perfectly. It just wasn't the job that mattered. The job that mattered was, don't let one bad weekend turn into a mess nobody signed off on. Nobody built for that job. They built for the demo job, draft a good-looking purchase order, and then scaled it thinking that it was the same as the real one. That's the whole mix-up.

7:24 In one weekend, the demo job and the real job looked identical for six straight weeks. They only came apart the one time nobody was watching, which is exactly when you find out which job you actually built for. So how do you catch that gap before Monday morning finds it for you? There's a rung system hiding inside that story already, and it's worth naming. Rung one is observed. The agent gets read-only access, drafts what it would do, and a human does every actual action. You're not testing whether it's clever, you're building a record of how often its draft matches what a human would have actually done or decided.

8:17 Rung two is supervised. Now, it can write at this rung, but every action still routes through one human click before it gets committed. Watch closely here, because this is where it quietly breaks. When the agents write, ninety sometimes running, that click soon becomes a rubber stamp. It stops being a review. It becomes a reflex. Rubber stamping is unattended mode with extra steps and someone else to blame. Avoid that trap. Rung number three is unattended. It acts alone inside a fence. That's the rung everyone wants on day one, and it's the rung you should reach dead last.

9:09 Before anything gets unclipped from rung two to rung three, there's exactly one question that matters, and it isn't, is it accurate? You already know that from rung one. The question is, what's the worst thing one bad unattended run can do with the tools that it's holding? That's the leash test. You scope the tools, not the prompt. Only reorder normal quantities is a sentence, and sentences are suggestions. A hard dollar ceiling built into the ordering tool itself is a wall, and walls should be within the wall-- the rollout.

9:58 And don't forget to build the skill switch on day one, not at six AM Monday after everything has happened. One action doesn't need the agent's cooperation, and it falls back to the boring process that was still warm the whole time. Now, what about this job versus demo? You know, that, that phrase, the job versus the demo, it isn't really mine, by the way. I stole it from myself several weeks earlier when I was writing about a completely different mess, a dashboard. Picture it, a deployment that's only two months old.

10:40 The dashboard's a wall of green. Ninety-four percent accuracy is being hit. Response time's under a second. Usage climbing every week. Everyone who built it is proud, and every single one of those numbers is measuring the model, not the job. So here's the swap that actually matters. It's called the funding test. And that-- In, in one sentence, I'll tell you what that is. For any number on that dashboard, ask whether it's moving the wrong way. And if it is moving the wrong way, would that actually change a budget decision?

11:22 Ninety-four percent accurate against what benchmark? And what happens to the other six percent? That's the exception rate, and it's the one that tells you the model's drifting while the demo stat stays flattering. Usage climbing isn't the same as the work getting done. That's task completion. And high usage with low completion rates just means that people are poking at a tool that doesn't quite work and then doing the job themselves anyway. Sub-second response time? Doesn't matter. If a human spends nine minutes double-checking the answer, that's cycle time.

12:07 Start to actually trusting it, done. Saves 20 hours a week, estimated by the person who pitched the project. That isn't a number, it's a hope wearing a number's clothes. The real one stat is cost per unit measured by somebody who didn't build the thing. Demo metrics, though, aren't fake. They're just meant for the engineering channel, not the board deck. The shops getting fooled aren't measuring less or the wrong things, they're measuring the one wrong thing confidently in green. Now, back to the people who told me they're mystified and don't know where to start because this next part isn't really about AI at all, and that's exactly the point.

12:59 I went back through a stack of, of different implementations that went sideways this year, and not one of them failed for a technical reason. Every postmortem blamed the platform, blamed the model. The platform was fine. What actually happened was a decision was made by default because making it on purpose felt like friction. Those all-at-once cutovers because phasing takes longer and everybody wants to be done. The scope that only ever grows because saying, "Not in this phase," out loud is uncomfortable, so it goes unsaid.

13:44 The data nobody owned because IT assumed it was a business call, and the business assumed it was IT's system. You also have knowledge that lives in exactly one person's head because getting documentation is always after go-live work, and after go-live is always the next project, so it doesn't get done. None of this requires you to understand a model, a token, an embedding, anything related to AI. It's the same four traps that have sunk ERP rollouts for 20 years. It's just wearing a new jacket. If you felt mystified by AI specifically, some of that feeling is just actually ordinary project discipline you already know how to do, dressed up in unfamiliar vocabulary.

14:42 That's also called the friction test. Find the decision getting made by default because nobody wanted to be the friction, and make somebody own it out loud before it gets expensive. Which brings me back to the two questions from the top of the show. Where's the friction? And would fixing it move a number? There, there's actually proof this year that the honest answer isn't exotic. Across industrial distribution, the AI wins landing the hardest right now aren't in some flashy warehouse robot. They're at the sales assist layer, quote generation, order support, the stuff that was already visibly, obviously slow.

15:34 MSC Industrial Supply crossed a billion dollars in quarterly sales, with digital continuing to grow as a share of that total. Global Industrial posted 9.2% quarterly sales growth and tied it explicitly to digital investment in customer retention. That's real revenue. Not a pilot, not engagement, revenue. The number that tells you it's going to last isn't the exciting one. Average order value jumping is the headline, and headlines are demo metrics, a launch bump that will fade. The number that's actually a job well done is the plateau, a steady weekly floor of traffic that holds, a repeat order cycle, time that keeps shrinking, revenue that compounds instead of spikes.

16:39 That's called the plateau test. You have to ask yourself, is the win a spike or is it a habit? Fireworks are impressive for exactly one evening only. That's a spike. A pilot light heats the house continuously all winter. That's a habit. And here in Michigan, having a habitual pilot light is very reassuring. The people running those portals didn't start with a grand AI strategy. They started exactly where the friction already visibly was, the ordering process everyone already complained about. And they proved it before they scaled it, which is incidentally still the name of this show.

17:32 [electronic sound] A few small things on my radar this month. Fast, because each one's a paragraph, not a full segment, and really a paragraph is honestly what they're worth. Number one, every vendor claims their ERP co-pilot is cutting ticket resolution time without cutting head count, but nobody's shown me the ticket count. Until somebody does, that's not a stat, it's a testimonial, and a testimonial is just a story wearing a blazer. Number two, AI committees are multiplying across mid-market manufacturing.

18:10 A committee that said no to something is doing a job. A committee that's approved everything it's ever seen isn't governing anything. It's a very well-catered rubber stamp with a standing calendar invite. If yours hasn't turned anything down yet, ask what exactly it's there to prevent. Number three, I keep seeing AI pilot used for what is functionally a rules-based automation script with a chatbot bolted on for the demo. Calling that an AI pilot is calling a stapler industrial fastening technology.

18:51 Not technically a lie, still, it's strange to put on a slide. Number four, outcome-based pricing sounds great. Pay only when the AI delivers, right up until you read what the contract actually calls delivered. I've seen definitions loose enough that the vendor technically wins even when you don't feel like you did. Vague enough is its own business model. Number five, human-in-the-loop used to mean we're not confident enough in this yet. Now, it's on the slide as a selling point. Either the market got smarter or marketing found a way to sell caution back to us and a markup.

19:40 I'd like to believe the former. I'm not betting the newsletter on any of it. Sixth and the last one, digital twin is having the loosest year of any term in this industry, including agentic. Half the ones I've seen are a dashboard with a floor plan graphic and some real-time numbers pinned to it. Not a simulation of anything, nothing you could actually run a what if scenario against. That's not a digital twin, though. That's a screensaver that knows the Wi-Fi password. If you can't ask it, what happens if line three goes down for six hours and get some real answers back, you bought a picture, not a twin.

20:31 That's the list. None of those six earn a full segment on their own, yet. They're more like the check engine light than the accident report. But give it six months and at least a couple of them will have either proven me right or made me look foolish on a schedule entirely of their own choosing. I'll report back either way because I was wrong about the committee thing is still a more useful sentence than most of what gets said in an actual AI committee meeting. Oh, and, uh, one more thing worth flagging this month because it's the industry-wide version of everything we just walked through.

21:15 RSM's Middle Market AI Survey polled one hundred and twenty-nine manufacturing companies in twenty twenty-six, and the results read like a highlight reel for tonight's argument. Fifty-six percent of manufacturers say they're using agentic AI today. Ninety-eight percent report satisfaction with the value it's delivered. Seventy percent of those, very satisfied. That's the confidence half of the story, and it's a lot of confidence. But here's the other half. Among the manufacturers whose pilots landed at only, uh, moderate or limited success, fifty-one percent blamed data quality, forty-three percent blamed integration challenges, thirty-four percent blamed security or compliance concerns, thirty percent blamed unclear ROI, another thirty percent blamed plain old adoption resistance or corporate speak for the humans didn't want to.

22:26 Every one of those is the same mix-up wearing a survey question costume. The demo looked ready, but the job wasn't. And here's the number that everybody who told me they feel mystified by, uh, 'cause this is the one that really actually explains it. Eighty-eight percent of manufacturers say they've got the right in-house AI expertise, but only twenty-two percent said their AI investment is concentrated on upskilling their own people. If you feel confused, that's not a personal failing. Four out of five companies you're competing against haven't concentrated their AI money there either.

23:16 RSM's Ryan Farlow put it about as plainly as a research analyst is allowed to. AI can't just sit on top of how work already happens. If it isn't built into the actual workflow, adoption and value both stay limited. That's the friction test again, running at the scale of an entire industry. Now, coming up next month, that eighty-eight versus twenty-two number is going to bother me until I write about it properly. If nearly nine in ten manufacturers think they already have the right AI talent in the building, but barely a fifth of them are concentrating investment on building it, who's really holding the leash at two in the morning when something goes sideways?

24:09 That's shaping up to be an August newsletter issue. If you've got a story about who's actually in the loop on your floor, respond to any issue I read every one. This episode proves that a real story beats a hypothetical one and means I'd genuinely rather build next month's discussion around yours. To wrap this up, four stories, four tests, one mix-up underneath all of it, mistaking the demo for the job. It happens to a Friday afternoon reordering agent and to a mystified reader who doesn't know where to start for the exact same reason.

24:54 Nobody stopped to ask which job they were actually looking at. If you take one thing from this episode, stop asking what AI can do. Ask where the friction already is in a job you already understand, and whether fixing it would move a number anyone would actually defend. Everything else, the leash, the metrics, the scope, the plateau, is just how you prove you got that first answer right. Full versions of tonight's four stories and the citation for the RSM survey are linked in the show notes. Check my math instead of just trusting the guy talking into a microphone, which, if you've stuck with me this long, is kind of the whole point.

25:42 Here's what I actually want from you. Where have you seen a demo win the room, and then quietly failed to do the job? Or where's the friction you already know about that you still haven't pointed anything at? Reply to any newsletter issue. I read every one, and I don't know where to start is exactly the kind of reply I want. That's next month's material, not a disqualifier. Plant Floor to Cloud, the newsletter, goes out every Tuesday. This podcast show runs monthly right here and wherever you're listening.

26:21 Subscribe, and if somebody on your floor is quietly confusing a demo for a job, send them this one. I'm Tony Gandek. Go hug your ERP. Talk to you next month.