Your AI Worked. It Still Didn’t Ship. | Plant Floor to Cloud Ep. 2

I'm Tony Gandick, and this is Plant Florida Cloud, the podcast. The mic's sometimes in shot, and I'm occasionally reading off of a script. If the vendor pitching you the next platform won't show you their seams, I at least won't hide mine. Last month, I ended this show by telling you a number was going to bother me until I wrote about it properly. 88% of manufacturers say they have the right AI expertise in the building. 22% are actually funding it. The same respondents, two separate questions, so don't go adding those up.

And I asked who's really holding the leash on AI at two o'clock in the morning. I spent August writing about it, four issues, and I want to be straight with you about how that went because I did not find what I expected to find. I expected a skills gap, people who need training but aren't getting it. What I actually found was worse and more interesting at the same time. Most companies don't think they have this problem at all. They're not underfunding something they know is broken. They're confident.

They're satisfied with their results, and the bill is quietly coming due somewhere they aren't looking. This episode, as always, has one theme all the way through. We are buying AI as a technology purchase, when in reality it's a capability purchase. The software technology will show up on the delivery date. The capability to support it, though, has to be built, and almost nobody is buying it. I've got a story that cost me three months to learn that. But before we get into it, quick context if this is your first time here.

This is the monthly companion to Plant Florida Cloud, the newsletter. Same charter as every Tuesday issue, mid-market manufacturing, practitioner-level detail, zero vendor hype. I go for about twenty-five minutes, some months longer, some months shorter. The newsletter tells you what happened each week. This show pulls one thread through the entire month. Last month's thread was mistaking the demo for the job. This month's podcast runs just underneath that one, and it's about money. Here's where we start.

[upbeat music] Now, before I get into the story, one question, and unlike last month's two, this one you should be able to answer in about fifteen seconds. Pull up your last AI approval, the requisition, the statement of work, the document of understanding, whatever your shop calls it. Now, find the line for training the people who have to run the thing. What percentage of the total was it? If you can't find the line at all, I think you already know where this episode is going. And if that stung a little, good.

You're in good company. It stung me too, and I'm the one who wrote four issues about it. And now the story. We could check the answer, not the reasoning. And here's what happened, and this is my own recollection of events, not a story I heard secondhand. A while back, we built an early warning capability on the quality side of the business. It read incoming customer sentiment, worked through the freeform text where the real complaints live, and watched patterns across resolution steps. What it was hunting for was the signature of a part starting to fail out in the field before rising claim volume would have made it obvious to anyone watching the normal way.

It existed for a reason. The business had already lived through a significant quality event. That event was why there was a budget, why there was executive attention and approval, and why nobody needed convincing that seeing the next one coming sooner would be worth something. So we tested it the way that you're supposed to. We took two years of historical data, including that original event I just mentioned, which the model had never been shown, and asked whether it would have caught it. It did.

And that first version of the model never reached production. It languished in proof of concept phase for a little over three months, and it was stopped there. Now, if you're waiting for me to tell you that the model was wrong or the vendor oversold it or the data was dirty, it was none of that. The thing worked, and that's why I'm still thinking about it, because the answer to why is simpler than you think. Two things can be true at the same time. A product can work and still not make it past that first proof of concept stage.

We thought that the back test against historical data was the finish line. Show that it can catch a real event and the argument's over. But the argument was just starting, and the questions that ended it were not the ones we had prepared for. Questions like How did it arrive at that? What was its scoring method? Was that a lucky guess or an actual signal? And who exactly is going to explain that to somebody who asks? We had answers to none of them in the form the business team could use. And here's what took me the longest to accept, and, and I'd ask you to sit with it for a second because it's really the load-bearing idea of this entire episode.

The back test validated the answer. It did nothing at all for the reasoning. Those reasons are two different things, and I had quietly assumed that the first would stand in for the second. I said it should be simple, but really it isn't. A correct result produced by a method you can't articulate isn't evidence, it's one result. It's not a trend. It's not a pattern. We had exactly one of them, one event caught once. Statistically, that's very close to nothing at all. If the model found it by reading real signal in complaint language, we'd have seen this outcome.

If it latched onto some fluke correlation that happened to line up that one time, we'd have seen this outcome, too. From outside the black box of AI reasoning, a working model and a lucky one, they look identical. The quality and field teams understood that before I did. And I want to be careful here because the lazy version of this story is the business people didn't get it. That's not what happened. They weren't obstructionist, and they weren't technophobic. They wanted it to work, but they also validate findings manually as routine, and they did that throughout this effort.

Verification was never the gap. They were pointing at something I walked right past. The team can't tell the difference between this system being right and this system being fortunate, and they are the people who are going to be asked by everyone else for justification. So you might be thinking, "I don't understand." Let me give you a perfect analogy. You remember the telephone game? One person whispers a sentence, it goes around the room, and by the time it comes back again, it's turned into something else entirely.

Not because anyone lied, just because the meaning erodes a little bit every time it changes hands. That's what happens to a model's output inside a company. Most people never notice because they only ever watch the first handoff. But think about the range. A recommendation engine suggesting a related part only travels one hop. The model says it, a buyer sees it, scrolls past it if it's wrong. Nobody downstream ever has to repeat it. But now climb a level. A quality flag says this part may be failing in the field.

It doesn't stop at one hop. It goes to a manager, then their boss, then maybe legal counsel, maybe a customer on the phone, through people who weren't in the room when it was built, retelling something, an event they didn't fully understand to someone else who understands it even less. That's the telephone game running completely against your org chart, and the finding degrades at every stop unless something or someone stops it. And there are actually three things that can be used to stop it. Number one, concrete data underneath the finding.

Number two, an explanation for how it got there. Or number three, somebody in the chain of command who can actually communicate both without losing the room. Take any one of those away and the message that reaches the end of the line isn't the message the model produced. It's whatever survived the retelling. Call it a, uh, defensibility test because that's really what it's asking. Can the person accountable for acting reconstruct the reasoning for an audience you will never meet? If the answer is no, accuracy is beside the point.

You've built a model that's right at the source and wrong by the time it matters. The alert becomes something that gets acknowledged and not acted on, which is the worst outcome available to you because now your organization is generating consequential findings that nobody is going to move on. Talk about crying wolf. And nobody down the chain is being unreasonable when that happens. Acting on one of those findings means pulling inventory, opening an investigation, putting your name on a recommendation that's going to get examined.

That's expensive, and that's what it means to be the one left holding the phone. Nobody wants to be holding the phone on a message that's already changed three times before it reached them. Now, the argument that's going to be made that actually changed my mind on this wasn't technical at all, and it's the part I'd want any technical leader listening to this podcast to hang onto. Remember this above all else If your brand promise is reliability, if the whole proposition is that your customers can trust what you put in the field as the manufacturer, then acting on a finding you cannot explain means spending trust you cannot account for and really cannot afford to lose.

Be wrong once publicly on grounds you can't defend, and you've damaged the exact asset the company is built on. That's far worse than the problem you were trying to prevent in the first place. So explainability isn't a nicety, it's not an option, and it isn't a compliance checkbox. Explainability is a brand control. It's the mechanism by which a company earns the right to stand behind a decision, and a business whose promise is trustworthiness structurally cannot make decisions it can't stand behind.

That's just one August issue. Here's the second one quickly because it's the other half of the same shape. Ask most IT leaders who can kill a live AI project and you get a pause and then probably an org chart answer. "Well, ultimately the steering committee," or, "I guess whoever sponsored it." Both are technically true and really functionally useless. A steering committee is a quorum, not a decision maker, maker, single, and the sponsor who has the worst possible conflict of interest for that specific call because they're also the person who has to explain to their boss why the thing they championed at the beginning didn't work in the first place.

That's not a character flaw. It's structural. It's self-preservation. Ask somebody to write their own bad performance review, and most people reasonably will delay. The project doesn't die. It just stops getting discussed in the room where it would have to be killed s- and starts limping along on residual budget and inertia. Give that a name. Call it the standing test, and it only has three parts, which is exactly why it almost never happens by default. Part one, named at kickoff, not after trouble starts.

Part two, not the sponsor, somebody whose performance review doesn't depend on the outcome. And part three, tied to a pre-agreed threshold, not a vibe. We'll know it when we see it is not a threshold. The reason my early warning story ended at three months instead of 18 is that somebody had the standing to stop it. That's the only thing that separates a clean stop from a zombie. The work itself didn't stop though, let me be clear. The model is still evolving, the way anything trained in customer language always has to.

What stopped cleanly was that first attempt at running it unsupervised. All right. Three stories last month, I covered three different looking failures. One, a chatbot that passed every test and got rejected because buyers wanted a part number and a delivery date, not a conversation. Two, projects nobody can kill because nobody was ever given the authority. And three, a model that was right and never shipped because nobody could explain it. Wrong shape, no off switch, no explanation. I spent most of August treating those as three separate problems, but if you look closely, they're not, not really.

They're one problem. It's just wearing three different costumes, and the survey data names it. 88% of manufacturers agreed they have the right in-house AI expertise. Only 22% said their AI investments are concentrated on training or upskilling internal talent. Nearly nine in 10 believe the capability is already in the building. About one in five is paying to put it there. Same 129 people answering two different and separate questions. They were never going to add up to 100. So if you're doing that math in your head right now, that's exactly why I'm saying this out loud.

But let's be precise about that 22 because it's easy to overstate, and I don't want you repeating a number wrong on my account. It does not mean only 22% do any training. It means that among manufacturers investing in AI, about a fifth have concentrated that investment on their own people. The rest goes to AI software and embedded solutions, about 61%, data platforms and infrastructure, about 45%, AI research and innovation, 44%. Now, look at that ranking honestly for a second because I don't think that those are bad decisions.

Every item on that list is defensible in a budget review. Every item on that list is also a thing. And things are easier to fund than people because a thing has a vendor, it has a quote, it has a delivery date, and it was also given a demo that was put in front of a committee. People enablement has none of that. It's a line item with no artifact at the end of it, so it loses, not because somebody weighed it and judged it unimportant, but because it never competed on equal footing. That's a procurement problem wearing a training problem's clothes, and it's why, "We should invest in training," is the most agreed with and least funded sentence in this industry, which gives us the last two standards, and these are the constructive ones.

Call the first the requisition rule, enablement that goes on the same approval as the platform, not a follow-on phase, not a Q3 initiative, not we'll circle back after go live. It's a line inside the same document, funded or not funded, at the same moment. Because if the platform gets approved and the enablement doesn't, you didn't underfund training, you approved a system with nobody qualified to operate it, and that should read to you as an incomplete purchase. It's the same thing as if the hardware showed up without a power supply.

You can't have one without the other. And budgeted as a percentage of the build, agreed to before you know what the software costs, 10%, 15, whatever survives your finance function. Pick the number early because trying to decide afterward means deciding when there's no budget left to allocate. Pair it with the second standard, the named person. This is the one that makes it real. Train the quality team is unmeasurable. It's not quantitative, and nobody owns it. Here's the version of the standard with a pass and a fail.

By go live, this named pr- analyst can explain any flag this system produces to a skeptic, unaided. You have your named person, you have a specific date, and you certainly have no help from the people who built it. That's resiliency. And then this is where last month's leash and this month's kill switch shake hands. Make failing that standard a documented reason the project can be stopped, same standing as a missed usage number. Otherwise, the project ships anyway, dies quietly in six months, and the model takes the blame for what was actually a budget decision that was made a year earlier.

Now onto my quick hits. [upbeat music] These are a few smaller things on my radar, fast because each one's a paragraph, and a paragraph is really what they're worth at the moment. Number one, I keep seeing AI literacy show up as a corporate initiative, which usually means a 90-minute recorded webinar and a completion certificate or a checkbox. Literacy means you can read and write. If the training ends before anyone has to defend a decision the system made, that's not literacy. That's a fire drill.

Number two, the phrase the model is explainable is doing enormous unearned work in vendor decks right now. The question is, explainable to whom? Explainable to a data scientist is table stakes. Explainable to Ed the VP, who has to justify a decision to a room that didn't attend the demo, is a completely different product, and I have yet to see it priced separately. Number three, watch for pilots that get extended rather than decided. An extension is what happens when nobody has standing to stop it and nobody can defend continuing it, so everyone agrees to think about it for another quarter.

That's not patience. That's a decision being avoided on a recurring calendar invite. Four, somebody's going to sell you an AI center of excellence this year. Ask what its budget is set to for teaching people outside of it. If the answer is nothing, it's not a center of excellence. It's a center of the excellence, singular, located in that room and honestly staying there. Number five, I've started considering asking vendors one question in the first meeting, "Who at my company will be able to explain this AI solution to my CFO in six months without you on the call?"

The good ones have an answer ready. It's been a remarkably efficient filter and surprisingly cuts down on a lot of noise, cold sales calls, and unwanted emails. Now onto an industry pulse. [gentle music] I have one more figure, and this is the one that explains why none of this self-corrects. That same RSM survey, 129 manufacturing respondents, asked what actually is blocking broader AI deployment. Talent and skills gaps came in at 24%. It ranked below security concerns, below data quality, and below legacy integration.

So this isn't a case of manufacturers knowing they have a problem and underfunding it. Most of them don't believe they have this problem in the first place, and honestly, why would they? By their own account, it's going well. 98% reported satisfaction with the business value AI has delivered, 70% of those very satisfied. The high confidence, high satisfaction, but small enablement budget. And here's the thing, none of these three facts is ever going to correct the other two because nothing in standard reporting connects the three together.

The projects that stall this way don't fail loudly. They fail the way mine failed, quietly, in a proof of concept, with everybody agreeing it was interesting. There's one more number that closes the loop. 84% agreed that executive leadership is more enthusiastic about AI than employees are. Think about it. Enthusiasm at the top, funded as technology, uncertainty at the point of use, unfunded as capability. That gap is precisely where all three of my breakdowns happened, and I'm not exempting myself in any of this or from any of this.

On the early warning project, we spent our budget on scoring and correlation and text analysis, and close to nothing on the interpretation layer. As I said previously, two things can be true at once. We were the 88% and the 22% at the same time, and we didn't notice until it had already stopped. Coming up next month, I want to run the other side of this argument because I've spent four issues and now two podcast episodes telling you about things that didn't work, and that's starting to feel like it's an incomplete picture, or I'm really negative about this technology.

So next month, the projects that actually paid, and I'll tell you now, none of them were the exciting ones. The ones that returned the most money in my career was centralizing 30 years of files that were sitting in folders nobody had opened or could understand without a Rosetta Stone. There's another where the entire innovation was running a project inside a video game years before anybody had the vocabulary for remote calibration or meta glasses. Neither of those got a demo. Neither of them got a committee.

Both of them earned more than anything I've ever put on a slide with the word AI in the title. I'm also going to talk about the rule to govern freight that's still working after the inventor retired from the company, and he didn't get a stage at a tech conference to implement it. That's September's newsletters. If you've got a boring project that quietly paid for itself, I want to hear about it. Reply to any newsletter issue you find on the Plant Florida Cloud website. And finally, four issues, four tests and standards, one bill nobody paid.

Can the person who has to act explain it to a room you'll never be in? That's defensibility, ending the telephone game before it ever gets started. Does somebody have the standing to stop it, named before the trouble started? That's standing. Who has the enablement? Is it funded? The requisition rule. And is there a named person who has to be competent by a specific date? The named person standard. Underneath all four is the same thing, and it's not a training problem. It's a purchasing problem.

We have to stop buying AI the way you buy a machine, thinking that when it arrives, you simply install it, and it runs, and you're done. AI is not a machine you install. It's a capability you build, and capability does not arrive on a delivery truck. The software shows up on the delivery date. The ability to run it shows up only if somebody paid for it. As I sign off, remember, the four August issues and the RSM citation will all be linked in the show notes. Go check my math instead of trusting the guy talking into a microphone, which is still the whole point of this show.

And here's your call to action, what I want from you this month, and it's very specific. Go find your last AI approval and tell me what percentage of it was enablement. I don't need the dollar figure, and I don't need your company's name. I want the percentage, and I want to know whether you had to hunt for it. Reply to any newsletter issue. If enough of you send me numbers, that becomes an episode, and it'll be better than a survey because it'll be ours. Plant Florida Cloud, the newsletter, goes out every Tuesday.

This podcast show runs monthly right here and wherever you're listening. Subscribe, and if somebody at your shop is about to approve a platform with no line for the people who will run it, send them this podcast before they sign. I'm Tony Gandick. Go hug your ERP. We'll talk to you again next month. [upbeat music]

In this episode