For most of my working life, troubleshooting an SMB environment was a hunt for a single, knowable cause. Something was broken, and somewhere there was a reason. A permission was wrong. A DNS record pointed at the wrong place. A service had stopped. You worked the chain backwards, found the link that had failed, fixed it, and the problem went away. The same input produced the same output, every time. That was the quiet contract underneath everything we did. IT was deterministic, and our whole troubleshooting craft was built on that assumption.
That contract is now breaking, and AI is the reason. The more our clients lean on tools like Microsoft 365 Copilot, the more we find ourselves chasing problems that don’t have a single cause and don’t behave the same way twice. We’ve spent decades learning to solve deterministic problems. We’re now being asked to solve probabilistic ones, and most MSPs haven’t noticed the ground shift under their feet.
“It Worked Yesterday” Now Means Something Different
Here’s the scenario I keep running into. A client calls because Copilot gave them a wrong answer. It summarised a meeting and missed the one decision that mattered. Or it drafted a reply in Outlook that referenced a document the user swears they never mentioned. Yesterday it was brilliant. Today it’s confidently wrong. Nothing changed on your side. No update shipped. No setting moved.
Under the old model, “it worked yesterday and not today” was a clue. It told you something had changed, and you went looking for the change. With AI in the mix, that same sentence tells you almost nothing. Large language models are probabilistic by design. The same prompt can produce a different response on Tuesday than it did on Monday, and that’s not a bug you can ticket your way out of. It’s how the technology actually works.
So when a client reports that “Copilot is broken,” your first instinct — find what changed — quietly fails you. There may be nothing to find. The behaviour you’re chasing isn’t a fault in the wiring. It’s variance in the output. And variance doesn’t sit still long enough to be caught with the tools we’ve always used.
The Cause Isn’t Always in the System
The harder adjustment is accepting that the problem often isn’t technical at all. When Copilot returns a poor result, the cause is frequently the question, not the code. A vague prompt, missing context, the wrong document open in the background, permissions that quietly scope what Copilot can and can’t see — these shape the answer far more than any registry key ever did.
I had a client convinced Copilot in Teams couldn’t read their project files. The real issue was that the files lived in a SharePoint site the user didn’t have access to, so Copilot, correctly, never touched them. The system was working exactly as designed. The human’s mental model was the thing that was broken. There was no error in any log, because there was no error. Try writing that up in a standard ticket resolution.
This is the part that unsettles seasoned engineers. We are trained to distrust “user error” as a lazy diagnosis. But with AI, the boundary between the tool and the person using it has genuinely blurred. The quality of what Copilot produces is now a function of context, phrasing, data access, and the user’s own clarity of thought. Half of real-world “AI problems” are actually grounding problems — Copilot simply wasn’t given the right material to work with. You can’t fix that with a script. You fix it by teaching.
From Repair to Probability Management
So what does troubleshooting look like when certainty is gone? It looks less like repair and more like managing probability. Instead of asking “what’s broken,” you start asking “why is this likely happening, and how do we make the good outcome more likely next time.”
That changes the work in practical ways. You start checking what Copilot can actually see — the SharePoint and OneDrive permissions, the Purview sensitivity labels, the data the user assumes is in scope but isn’t. You look at how the question was asked, not just what the system returned. You reproduce the issue several times, because one bad answer is an anecdote, not a pattern. You document tendencies rather than root causes, because a tendency is often the most honest thing you can record.
It also changes what you sell. The deterministic world rewarded MSPs who could find and fix. The probabilistic world rewards MSPs who can guide, set expectations, and shape how AI gets used across a client’s day. The value moves from the repair to the relationship.
Sitting With Uncertainty
None of this means our old skills are worthless. Plenty of SMB problems are still gloriously deterministic — a licence didn’t assign, a mailbox didn’t migrate, a Conditional Access policy locked someone out. Find it, fix it, move on. That work isn’t going anywhere.
But a growing slice of what lands in your queue now has no clean answer, and pretending otherwise only frustrates everyone. The MSPs who’ll do well from here are the ones who can hold two modes at once — the precision of the engineer and the judgement of an advisor who’s comfortable saying “here’s what’s most likely, and here’s how we improve the odds.” Learning to sit with that uncertainty, rather than fight it, might be the most valuable troubleshooting skill of the next decade. I’m still getting used to it myself.
CIAOPS 