AI Doesn’t Replace You, It Replaces the Previous AI

IT consultant in a modern office with older devices fading behind him in teal and amber light

Every time a new AI tool lands, someone asks me the same nervous question. Is this the one that finally replaces me? I understand the worry. But after watching the last couple of years up close, I think we’ve been asking the wrong question. AI isn’t really replacing you. It’s replacing the previous AI system.

Think about how quickly the tools themselves have turned over. The chatbot you were impressed by not long ago now feels clumsy. The prompt tricks you learned have been made redundant by smarter models. The agent you built in spring has been overtaken by something that does the same job with half the effort. The churn is happening at the tool layer, not the human layer.

The tools are the disposable part

Models get better, then get swapped. Agents get rebuilt. The harness around them, the connectors, the scripts and the guardrails, keeps evolving. I wrote recently about how one assistant changed the way I work in Finally, AI That Just Gets Things Done, and I’m under no illusion that it will be the last tool I rely on. Something better will arrive, and I’ll move to it.

That’s the pattern MSPs and business owners need to get comfortable with. Copilot in Teams, Outlook, Word and Excel will keep changing underneath you. Features will be renamed, merged and retired. Agents in Copilot Studio will be rebuilt as the platform matures. If your value is tied to knowing one specific tool, your value has a shelf life.

What doesn’t get replaced

Here’s the thing the hype tends to miss. Every one of those tools still needs someone who knows what good looks like. AI can draft the proposal, but it doesn’t know which client is about to walk. It can summarise the meeting, but it doesn’t know which decision actually mattered. It can close the alert, but it doesn’t know why that alert keeps coming back.

The lasting value sits with the person who brings:

  • Knowledge of the business, its people, its customers and its quirks
  • Judgement about what matters, what’s risky and what’s good enough
  • Workflows that are understood well enough to hand to whatever tool comes next
  • The trust of the people who rely on them

None of that lives inside a model. All of it transfers when you change tools. That’s the part worth investing in.

What this means for MSPs

For MSPs, this is good news, as long as you don’t anchor yourself to a product. I argued in AI Doesn’t Care About Your MSP Business Model that clinging to old ways of working is the real risk. The same applies to clinging to this year’s AI tool. Your clients don’t need you to be loyal to a platform. They need you to be the one who knows how their business runs and can plug in whatever tool does it best today.

Picture an MSP that documents a client’s onboarding process properly, in SharePoint and Planner, with clear steps and owners. When a better agent comes along, they simply point it at the same process. The workflow survives. The tool is the replaceable part. That’s a far stronger position than being the expert in one bot that’s obsolete by next year.

So stop measuring yourself against the AI. Measure yourself by how quickly you can adopt the next one, and by how well you understand the work it’s meant to do.

AI will keep replacing AI. Make sure you’re the constant it gets plugged into.

It’s Not the Model Anymore, It’s the Harness

An IT consultant at a desk surrounded by glowing connections linking a central light to many devices

For the last couple of years, most AI conversations I’ve had started the same way. Which model is best? Which one is smartest this month? Who just topped the leaderboard? I get why. The models were the shiny part. But I’m noticing the question changing, and I think it’s the right change.

The model is becoming the engine. What actually decides whether AI is useful in your business is everything bolted around it: the harness, the tools, the data it can reach and the rules it works within.

The engine isn’t the car

A brilliant model sitting on its own knows a lot about the world and nothing about your business. It can’t see your customer list, your contracts or yesterday’s Teams meeting. It can’t raise a ticket, update a Planner task or check a device in Intune. Without connections, it’s a very well-read stranger.

Microsoft 365 Copilot is a good example of where the value really sits. Microsoft describes Copilot experiences as differing by data grounding, integration depth and licensing, with work grounding coming through Microsoft Graph. Notice what that list doesn’t lead with. It’s not the model. It’s what the model is plugged into.

Picture two businesses using the same underlying model. One just chats with it. The other has it grounded in SharePoint, connected to their line-of-business system and able to run a Power Automate flow. Same brain, wildly different outcomes.

Tools are the new skills

This is why I’m paying so much attention to the Model Context Protocol. MCP is an open-source standard for connecting AI applications to external systems. In plain terms, it’s a common plug. Build a connector once and many AI tools can use it.

Microsoft has already brought it into Copilot Studio, where you can extend an agent with tools using MCP. That changes how I think about building agents. The hard work is no longer picking a model. It’s deciding which tools an agent gets, what data it can see, and what it’s allowed to do with them.

I touched on a version of this when I wrote Why developers don’t own this one. The value of an AI tool at the command line came from what it could reach and run, not from the model’s raw IQ.

What this means for MSPs

If you run an MSP, this is good news. Model choice is turning into a commodity decision that vendors will keep changing underneath you. The harness is where your expertise lives. That’s the bit you can design, secure and support.

Here’s where I’d focus:

  • Grounding: is the data Copilot reaches current, permissioned properly and worth trusting?
  • Connectors: which systems should an agent touch, and which should it never see?
  • Guardrails: what can an agent read versus actually change, and who approves the risky steps?
  • Identity and audit: can you tell, through Entra and Purview, what an agent did and on whose behalf?

That last point matters. Every new tool you connect is also a new door. Exposed or poorly authenticated MCP servers have already shown up in security research I covered in the CIA Brief 20260516. A great harness is a secure harness.

Stop asking which model

I’m not saying models don’t matter. Better reasoning helps. But the gap between good models is shrinking, while the gap between a well-connected agent and a lonely chatbot is enormous.

So next time someone asks me which AI is best, I’ll answer with a question. Best at what, connected to what, and trusted to do what? Get those answers right and the model almost picks itself.

Who Owns the Agent When the Employee Walks Out the Door?

A professional leaving an office tower at dusk, glowing threads trailing from her briefcase back to the building

Here’s a question I don’t think many businesses have asked yet. When you hire your next person, will they be allowed to bring their own agents with them? And when someone leaves, will you let the agents they built in your business walk out the door too?

For years we’ve treated a worker’s toolkit as personal. Their know-how, their shortcuts, their way of working. Agents blur that line badly. An agent can hold skill, but it can also hold memory, data and relationships. That makes the answer far less obvious than it first appears.

The toolkit versus the tenant

I think the cleanest way to start is to split an agent into two parts.

The first part is the pattern. The prompts, the instructions, the way someone has learned to break a job down and hand it to AI. That’s skill. It lives in their head as much as anywhere, and I don’t believe a business can or should claim it. A good tech who has learned how to direct agents well will carry that ability to their next role, the same way they carry their troubleshooting instincts.

The second part is the data. An agent built in Copilot Studio inside your Microsoft 365 tenant, grounded on your SharePoint libraries and connected to your systems, belongs to the tenant. So does what Copilot remembers about how a person works. Microsoft stores Copilot memory in a hidden folder in the user’s Exchange mailbox, inside your tenant, and admins can find it with Purview eDiscovery. That’s a pretty strong hint about where ownership sits.

I wrote recently about the mystery of personal SharePoint skills, which end up stored in a user’s OneDrive. Personal in name, but still sitting in the company’s tenant. Expect a lot more of that.

Memory is where it gets messy

Picture a consultant who has spent two years working with an agent. It knows her clients, their quirks, which contracts are touchy and which projects ran late. Now she moves to a competitor. Should that memory go with her?

My view is no, and not because the business is being greedy. That memory is tangled up with client data, confidentiality obligations and, in many cases, data residency requirements. You can’t neatly pull “how she works” out of “what she knows about our customers”. Until the tools can genuinely separate the two, the memory has to stay.

Treat BYO agents like BYOD

Bringing your own agent in the other direction feels a lot like the early days of bring-your-own-device. The answer then wasn’t a flat yes or no. It was policy, identity and control. Agents need the same treatment. Whose identity does it run under? What data can it reach? Where does what it learns end up? If you can’t answer those, the agent shouldn’t touch your tenant.

What I’d do now

None of this is legal advice, and your lawyer will have views. But there are practical steps any SMB or MSP can take today:

  • Write a simple agent policy covering who can build, bring and share agents.
  • Keep an inventory of every agent, with a named human owner.
  • Add agents and AI memory to your offboarding checklist, right next to mailbox and device handover.
  • Update employment contracts to say what’s portable and what stays.
  • Decide upfront that skills and prompt patterns travel, while data and memory stay put.

The people who learn to work well with agents will be worth more wherever they go. That’s fine. Let them take the skill. Keep the data. Just make sure you’ve written down which is which before someone hands in their notice.

Your Agents Will Outnumber Your People

An IT administrator facing a vast glowing wall of identity nodes multiplying outward from a few people

Most of the businesses I work with know roughly how many user accounts they have. They know who joined last month and, if they’re doing it right, who left. Ask the same people how many AI agents are working in their tenant and the room goes quiet. That silence is the problem I want to talk about, because I think agents are about to become the fastest growing identity population in every business.

Agents are identities, not features

An agent that reads mail, files documents in SharePoint, updates a Planner board or posts into Teams is acting inside your environment. It needs access, and that access has to be granted, scoped, watched and eventually removed. In other words, it needs to be managed like a user.

Microsoft has made this explicit. Microsoft Entra Agent ID, now generally available, gives AI agents their own identities in Entra. Each agent identity is created from an agent identity blueprint, and that blueprint must have a sponsor: a person or group who is accountable for the agent. Conditional Access and Entra ID Governance are being extended to cover agent identities too. I like that design, because it forces the question every business should be asking anyway: who owns this thing?

I’ve been living this myself. I recently wrote about running Microsoft 365 connections headless for autonomous agents, and the first thing you realise is how much power a non-human identity can hold.

The maths changes everything

Here’s the bit that worries me. Picture a 20-person firm. Today that’s 20 accounts to look after. Now imagine each person runs three to five agents: one triaging their inbox in Outlook, one building reports in Excel, one built in Copilot Studio to answer client questions, another chasing approvals in Power Automate. That’s potentially 60 to 100 agents, with no new hires. The headcount stays flat while the identity count multiplies.

We already struggle with user sprawl. Agent sprawl will arrive faster and quieter, because creating an agent feels like a productivity win rather than an IT event.

Where it goes wrong

The failure modes are predictable:

  • Orphaned agents. A staff member leaves, their account is disabled, but the agents they spun up keep running with nobody watching.
  • Over-permissioned agents. It’s easier to grant broad access to get something working than to work out the least privilege it actually needs.
  • No inventory. Nobody can list what exists, what it touches or why.
  • Hidden cost. Agents consume licences and capacity, and the bill grows without a clear owner.

These are the same identity hygiene issues we’ve fought for years, just bigger and faster.

What I’d do now

For SMBs, and especially for MSPs looking after them, I’d treat agents exactly like people from day one:

  • Every agent has a named human owner. No owner, no agent.
  • Use a consistent naming convention so you can tell what an agent does at a glance.
  • Keep an inventory and review it, just like an access review for staff.
  • Grant least privilege and apply Conditional Access where you can.
  • Add agents to your offboarding checklist. When someone leaves, their agents get reassigned or retired.
  • Monitor activity, and use tools like Purview and Defender to keep an eye on what data agents touch.

MSPs have a real opportunity here. Customers will need someone to count, govern and tidy up their agents, and that’s work that fits neatly beside the identity management we already do.

The businesses that win with AI won’t be the ones with the most agents. They’ll be the ones who still know what every agent is doing and who answers for it. Start counting now, while the number is still small.

Who Is the Software For Now?

Cartoon of a person relaxing with coffee while app windows around them work on their own

Picture asking Copilot in Teams for a status report on a client project. It goes through the project files in SharePoint, pulls numbers from an Excel workbook, checks which Planner tasks are overdue and drafts a summary to send from Outlook. You never open a single one of those apps.

That gets me thinking. If we don’t need to open the apps, who exactly are they for now?

Is software going headless?

“Headless” usually describes a system with the engine but no front end. You talk to it through an API rather than a screen. What I’m seeing with Microsoft 365 Copilot, agents and Copilot connectors looks like the same idea applied to everyday work. You give the AI a goal, and it goes off and talks to the tools directly.

Take it a step further. Bring in a line-of-business system through a connector or an agent built in Copilot Studio, and that report could include CRM or ticketing data too, without logging in to anything. The tools are still doing the work. Nobody is looking at them.

So is the SaaS product we pay for every month slowly becoming a tool for AI rather than for people? For a growing slice of routine work, I think yes. I’m not convinced it’s the whole story, though.

What it means for the people using the tools

If you’re a creator or knowledge worker, the skill shifts from knowing where every button lives to knowing what to ask for. And knowing what good looks like when the answer comes back.

That sounds liberating, and it is. But it means people may understand their own systems less. When the report is wrong, someone still has to know which spreadsheet fed it and why. If nobody ever opens the tool, who notices when the data inside it has rotted?

And humans aren’t leaving the screen entirely. We still need interfaces to review, approve, handle the exceptions and make the creative calls that don’t fit a prompt. I don’t want an agent sending a client proposal without me looking at it first. The UI doesn’t disappear. It shrinks down to the moments where judgement and trust matter.

What it means for the people building the tools

This is where I think the bigger shake-up sits. Plenty of SaaS companies have built their business on a polished interface, per-seat pricing and the stickiness that comes from people living in the app all day. If an agent is the main user, what’s a seat? And how much is a beautiful dashboard worth if nobody looks at it?

My guess is differentiation moves underneath the interface. It shifts to the quality of the data, how good and open the APIs are, reliability, and how well permissions and security are handled. When AI is acting on my behalf, I care a lot about whether the tool respects who is allowed to see what. Trust becomes the product.

Vendors that treat their API as an afterthought could find themselves invisible. The tool might work perfectly but never get picked by the agent.

So, where does that leave us?

I don’t think the apps vanish. I do think the people who build them need to work out who their real customer is. Maybe it’s the human, maybe the agent, and most likely both. For the rest of us, the job is learning to direct the work rather than do every click ourselves, while staying close enough to the detail to spot when it goes wrong.

So I’ll put the question to you. How much of your working day already happens without opening the app? And will the tools you rely on still matter if you never see them?

Stung by my own stupidity

If you aren’t aware, when you built or run anything new in Copilot Studio it will all be charged PAYG against Azure. To allow this you need to connect your Power Platform environment to an Azure subscription. You can find details on how to do that here:

https://blog.ciaops.com/2022/04/29/set-up-payg-for-power-platform/

Now with all that in place I built a new modern agent in Copilot Studio to answer M365 questions (called Sage) built using the new Github Copilot harness and using Claude Opus 5 as the LLM.

image

I then tested it a few times in ‘Preview’ and was happy that it was all working. I knew at this point, all that was going to cost me a few bucks because now even creation costs with the new Copilot Studio. All good so far and still in budget.

Next, I wired up a new Workflow in Copilot Studio to wait for a message to be posted into a Microsoft Teams channel, take that, post it to the newly created Sage agent, then take the reply from the agent and post it back into the same channel. Quick and easy to create. Job done, I thought.

Can you see the logic flaw yet? I certainly didn’t initially. In short, the workflow I created basically replies to every message posted into a channel. Ahem, those replies then trigger the agent to run again and post yet another message, which again triggers another message posting from the agent, and on and on. So, I had created an infinite loop.

My mistake was now running the workflow and calling the agent and posting into the Team every minute or so. I didn’t recognise my error for a few hours! Yes, hours. I estimate the loop I created with the workflow ran for about 3.5 hours in total. Ouch. When I finally realised upon checking back into the channel I immediately deleted the workflow to stop the race condition, however I knew I was going to pay for my mistake.

Fast forward a day or so when I have all the billing data available. Here’s what the results of my oversight were:

Screenshot 2026-08-21 073742

The error had cost me around AU$250. D’Oh!

All of this is always a learning experience, so now that I had understood the ‘bill shock’ amount I wanted to see what more billing information I could obtain about what had actually happened. I visited the Power Platform admin center | Licensing Copilot Studio, scrolled down to Top 5 agents and users, then selected View all agents which showed me this:

image

then when I drilled into my environments I can see:

image

and at the bottom you can see the autonomous consumption of 16,376.01 credits. If I divide that by the 3.5 hour run time I get 4,678.86 credits consumed per hour. If I then divide that by 60 to get the cost per minute I get 77.98 credits. Thus, each post to the channel in effect cost around US$0.78 which is about AU$1.20.

The detail also shows that creating the agent cost around 67.26 + 379.77 = 447.37 credit which is around US$4.50 and AU$6.95 to create.

I have now added notifications at 90% capacity like so in this admin console because they are not enabled by default:

image

I would expect, like the budget notifications from Azure, they are not immediate which makes avoiding costly mistakes harder when your error maybe racking up a few dollars per minute charges!

I accept full responsibility for my error and oversight and bill incurred, however I think there are some important learnings and observations here with the new PAYG billing for AI services. These in essence boil down to the fact that it very difficult to get a good understanding of exactly what your costs are in real time or prior. Typically, you need to wait a full 24 hours until all the billing data has been collected and by then you maybe up for thousands of dollars if you are not very careful.

Another observation is that if you make a logic error in your build you won’t find that until you look at your bill. I was lucky that I found mine after a few hours, imagine if it had run for more than 24 hours before the billing data alerted me? Ouch.

I believe this lack of immediacy an d visibility on costs is going to be a major barrier for adoption of PAYG agents in Microsoft 365, whether Cowork or the new Copilot Studio, especially in SMB.

image

Hopefully, we get to a point like we have with Github Copilot (above) where I can quickly and easily see my usage in the development environment (here Visual Studio Code). Without this type of spending certainty many business are simply not going to use what are fantastic AI tools to help their business. This risk of runaway costs is simply too great.

Another point that I want to reinforce here is that when you implement PAYG with agents you need to monitor your costs DAILY! This will be a big change for many MSPs who may occasionally go into a customers tenant to look at licensing monthly. If your customer has PAYG AI and you are responsible for managing these costs you need to keep an eye on this every single day to minimise what a single logic error could cost.

Ensure you enable all the alerting that you can when you use PAYG AI services, no matter where or whom they are from. Hopefully, doing this and my sorry tale here helps you better monitor your costs and avoid ‘AI usage bill shock’.

Your first Copilot Studio agent — and the bits the demo skips

MAI_111667a8d6c40e32

Everyone wants to build an agent right now.

It’s the demo that makes the room go quiet. You describe what you want in plain English, and thirty seconds later there’s a working assistant answering questions from your own documents.

Then the bill arrives. Or worse — it doesn’t, because nobody published the thing and it’s sitting in a maker’s personal environment doing nothing.

Here’s what I keep seeing. People treat an agent as a science project. They build one, screenshot it, post it on LinkedIn, and move on. The client never touches it.

That’s not a product. That’s a party trick.

If you’re an MSP, the agent isn’t the deliverable. The service around it is. Let me show you the bits people skip.

What is a Copilot Studio agent, really?

Strip away the marketing and an agent is three things: instructions, knowledge, and tools.

Instructions are the job description — what it does, the tone it takes, when to give up and send someone to a human. Knowledge is what it’s allowed to read: a SharePoint site, a public page, a handful of documents. Tools are the actions it can take, like logging a ticket or looking up an order.

That’s it. You’re not training a model. You’re scoping a very literal new hire.

And there are two doors into this. The Agent Builder baked into Microsoft 365 Copilot, and Copilot Studio proper. The choice between them comes down to audience and reach — a quick helper for your own Copilot chat, or a real one you publish out to Teams, a website, wherever your client’s people already work.

Step-by-Step: building your first agent

Portal only. No code.

Open the studio

Go to copilotstudio.microsoft.com and check the environment name in the top right. This matters more than it looks. Build in the wrong environment and you’ll be untangling it later. Pick the client’s environment deliberately, every time.

Describe what you want

You get a box. Type what the agent should do in your own words — that’s the whole guided, no-code start. Mine for a small-business help desk reads like this:

Help our staff with common IT questions — password resets,
VPN setup, who to call for hardware. Answer only from the
company IT knowledge base. If you're not sure, say so and point
them to the service desk. Don't guess.

Notice what’s missing? No model to pick. No temperature dial. No code. You’re writing a job description in plain English. And the load-bearing line is the last one: ask it once to admit when it doesn’t know. An agent that says “I’m not sure” beats one that bluffs with total confidence.

Add knowledge, narrowly

Point it at one good source first. The company FAQ. The IT policy site. Not everything you own — one trusted thing. Test it. Then add more. An agent fed the whole tenant on day one is an agent that confidently surfaces the wrong document.

Test like a real person

Use the test pane on the right. Ask the messy questions: “can’t log in”, “where’s the VPN guide”, “who do I call”. If it waffles, fix the instructions — not the model.

Publish it

This is the step everyone forgets. An agent does nothing until you hit Publish. And every change you make afterwards needs publishing again, or your users sit on the old version. Then add a channel — Teams and Microsoft 365 Copilot are the obvious two — and decide who’s allowed in.

By default it’s Authenticate with Microsoft, which means your tenant and no one else. Leave it there. “No authentication” hands a working link to the entire internet. For an internal SMB agent, that is never what you want.

The licensing question you answer first, not last

Here’s the bit the demos never mention.

Agents built in the Agent Builder inside Microsoft 365 Copilot ride along with the licence your client already pays for. The custom agents you build and publish in Copilot Studio consume Copilot Credits — and since September 2025 that’s the currency Microsoft meters them in. Pay-as-you-go, or prepaid packs.

“So I just describe it and it works?” Yes — and that’s exactly the trap. The describing is the easy part. The scoping, the publishing, and the consumption call are the actual job.

So before you build, settle one thing: is this an internal helper for a few Copilot users, or a published agent the whole company will hammer all day? The answer decides which door you walk through and whether there’s a consumption bill hanging off it.

Get that wrong and you’ve either over-built a toy or under-budgeted a product.

The point of an agent isn’t to look clever for ninety seconds in a demo.

It’s to take a repetitive question off a human’s desk and answer it the same way, every time, at 2am. A published agent earns its keep. An unpublished one is a screenshot.

Build one narrow agent, publish it properly, and put your name on the governance around it. If you’re not showing your clients this, someone else will — and they’ll be the ones charging for it.

Need to Know podcast–Episode 360

In this episode I’m joined by Shervin Shaffie from Microsoft to do a deep dive into Copilot Studio. That’s the service from Microsoft that allows you to create agents in a ‘low-code’ manner right inside the M365 environment. Shervin has some great Youtube content I highly recommend and provides some great insights and tips and tricks in this episode when working with agents in Microsoft 365. I’ll also brin you up to date with the latest Microsoft Cloud news. Listen along.

Brought to you by www.ciaopspatron.com

you can listen directly to this episode at:

https://ciaops.podbean.com/e/episode-360-shervin-shaffie/

Subscribe via iTunes at:

https://itunes.apple.com/au/podcast/ciaops-need-to-know-podcasts/id406891445?mt=2

or Spotify:

https://open.spotify.com/show/7ejj00cOuw8977GnnE2lPb

Don’t forget to give the show a rating as well as send me any feedback or suggestions you may have for the show.

Resources

Explore the tools, communities, and content mentioned in this episode:

Show Notes

Shervin Shaffie – https://www.linkedin.com/in/sherv/
Principal Copilot Engineer at Microsoft
Collaboration Simplified YouTube:
https://youtube.com/@collaborationsimplified
All About AI Podcast: https://cosi.pro/aipodcast

FY26 Q2 – Press Releases – Investor Relations – Microsoft

The Microsoft Copilot Data Connector for Microsoft Sentinel is Now in Public Preview | Microsoft Community Hub

Secure Boot playbook for certificates expiring in 2026

SharePoint Showcase highlights: Copilot and agents governance and security essentials for admins

What’s New in Microsoft 365 Copilot | January 2026 | Microsoft Community Hub

Upcoming Conditional Access change: Improved enforcement for policies with resource exclusions