Every time a new AI tool lands, someone asks me the same nervous question. Is this the one that finally replaces me? I understand the worry. But after watching the last couple of years up close, I think we’ve been asking the wrong question. AI isn’t really replacing you. It’s replacing the previous AI system.
Think about how quickly the tools themselves have turned over. The chatbot you were impressed by not long ago now feels clumsy. The prompt tricks you learned have been made redundant by smarter models. The agent you built in spring has been overtaken by something that does the same job with half the effort. The churn is happening at the tool layer, not the human layer.
The tools are the disposable part
Models get better, then get swapped. Agents get rebuilt. The harness around them, the connectors, the scripts and the guardrails, keeps evolving. I wrote recently about how one assistant changed the way I work in Finally, AI That Just Gets Things Done, and I’m under no illusion that it will be the last tool I rely on. Something better will arrive, and I’ll move to it.
That’s the pattern MSPs and business owners need to get comfortable with. Copilot in Teams, Outlook, Word and Excel will keep changing underneath you. Features will be renamed, merged and retired. Agents in Copilot Studio will be rebuilt as the platform matures. If your value is tied to knowing one specific tool, your value has a shelf life.
What doesn’t get replaced
Here’s the thing the hype tends to miss. Every one of those tools still needs someone who knows what good looks like. AI can draft the proposal, but it doesn’t know which client is about to walk. It can summarise the meeting, but it doesn’t know which decision actually mattered. It can close the alert, but it doesn’t know why that alert keeps coming back.
The lasting value sits with the person who brings:
Knowledge of the business, its people, its customers and its quirks
Judgement about what matters, what’s risky and what’s good enough
Workflows that are understood well enough to hand to whatever tool comes next
The trust of the people who rely on them
None of that lives inside a model. All of it transfers when you change tools. That’s the part worth investing in.
What this means for MSPs
For MSPs, this is good news, as long as you don’t anchor yourself to a product. I argued in AI Doesn’t Care About Your MSP Business Model that clinging to old ways of working is the real risk. The same applies to clinging to this year’s AI tool. Your clients don’t need you to be loyal to a platform. They need you to be the one who knows how their business runs and can plug in whatever tool does it best today.
Picture an MSP that documents a client’s onboarding process properly, in SharePoint and Planner, with clear steps and owners. When a better agent comes along, they simply point it at the same process. The workflow survives. The tool is the replaceable part. That’s a far stronger position than being the expert in one bot that’s obsolete by next year.
So stop measuring yourself against the AI. Measure yourself by how quickly you can adopt the next one, and by how well you understand the work it’s meant to do.
AI will keep replacing AI. Make sure you’re the constant it gets plugged into.
The first person to tell an MSP about a Microsoft 365 outage should not be the client who cannot open Outlook. I would rather a Grok Bot routine check status.cloud.microsoft for me, tell me when Microsoft posts an incident, keep watching while it is open, and tell me when it is marked restored. If customers might be affected, I want the same bot to draft a notice I can approve. It should not send anything on its own.
That is the whole job. Grok Bot is the monitor. The public status page is the source. A routine is the schedule. Everything else in this post – the admin centre Service health page, and the Graph API behind it – is useful context for later. It is not the starting point.
I am writing this on Friday, 13 November 2026, in Sydney. While I was putting the post together, the public page was already showing a service degradation: users may be unable to connect to multiple Microsoft 365 services, including Exchange Online, Microsoft Teams, SharePoint Online and Microsoft 365 Copilot. Microsoft had posted a current status, a previous status, and a time for the next update. That is exactly the sort of event I want Grok Bot to catch and stay with, rather than me discovering it in a ticket queue.
What Grok Bot is actually watching
Microsoft documents the public service status page as the place to check when you cannot sign in to the admin centre. The official write-up is How to check Microsoft 365 service health. The page is unauthenticated, so a Grok Bot routine can fetch it without a tenant sign-in. That is the point. If the admin centre itself is hard to reach, you still have a public source.
The site is a status page, not a full history. I checked what it actually exposes, rather than inventing an API. The page’s own code loads a current Microsoft 365 post from https://status.cloud.microsoft/api/posts/mac, and it publishes the same item as RSS at https://status.cloud.microsoft/api/feed/mac. There is a matching Power Platform feed at /api/feed/ppac, and the page points at Microsoft’s Azure status RSS for Azure. The Microsoft 365 post I fetched included a title, a status such as “Service degradation” or “Service restored”, timestamps, and a message with user impact, the current status, earlier updates, and a “Next update by” time. That is enough for a bot to notice a new incident, see whether the wording has changed, and notice when the status returns to available or restored.
One practical detail is worth knowing. The public page itself refreshes those posts on a five-minute interval. That is in the page’s own code. It is not a published Microsoft SLA for how quickly an update will appear. Microsoft often says when the next update is due inside the post. Grok Bot should treat that time as a promise to check against, not as a guarantee.
The public page is also broad. It tells you about a widespread issue Microsoft has chosen to post there. It does not tell you whether a particular tenant is affected, and it will not list the quieter advisories that show up in a tenant’s own Service health view. I will come back to that difference later. For the first routine, the public page is the right source because Grok Bot can read it with no extra permissions.
Set up a Grok Bot routine to watch and alert
You can do this on a dedicated status bot, or add the routine to a bot you already use for monitoring. I already use Grok Bot routines for a morning news brief, which I walked through in How to Use Grok Bot to Monitor MSP News and Security Updates. This is the same idea, pointed at one official page.
The app locations I will point you to are ones I have verified. Click the agent’s name in the chat header, or use Cmd+Shift+I, to open the per-agent info pane. Saved routines sit in that pane, under the computer preview. Check the timezone under Settings, on the General tab, opened from the sidebar account button at the bottom-left, or with Cmd+,. If you want the bot named and described clearly, the gear beside the X in the info pane opens the per-agent settings for avatar, name, title and description.
Create the routine in chat. Routines usually start within a few minutes either side of the time you set. If the run must start right on the minute, say “exactly” or “sharp”.
Here is the prompt I would use:
Create a routine called Microsoft 365 public status watch. Every 15 minutes during my Sydney business hours, check https://status.cloud.microsoft/ and the Microsoft 365 RSS feed at https://status.cloud.microsoft/api/feed/mac. You can also read the current post at https://status.cloud.microsoft/api/posts/mac.
Compare the current title, status and message with what you saw on the last run. If nothing material has changed, stay silent.
If a new incident or service degradation is posted, or if the status, title or message has changed, post an alert in this chat. Include:
- the title and current status
- the affected services, in plain English
- the current update, and what changed since last time
- the "Next update by" time, converted to Sydney time
- a link to https://status.cloud.microsoft/
Do not invent endpoints or details that are not on the page or in the feed. Do not sign in to a tenant for this routine. Do not email or message anyone else. If the page cannot be reached, say so once and try again on the next run.
Fifteen minutes is a starting cadence, not a law. The page is already checking about every five minutes, so you can tighten the routine if you want it closer to that. Outside business hours you might run it less often, then keep a closer watch once an incident is open. What matters is that Grok Bot compares each run with the last one, and only speaks when something has changed.
Keep watching during the outage
An alert is only the start. Once Microsoft has posted an incident, I want Grok Bot to stay with it. The useful work is noticing what changed in the update, not repeating the same paragraph every quarter of an hour.
Ask it, in the same chat, to tighten the watch for as long as the incident is open. I would tell it to check again at or just after the “Next update by” time, and otherwise on a short interval, and to report only when the current status text, the list of affected services, or the overall status has changed. If Microsoft slips past the promised update time with no new post, I want the bot to say that too.
A during-outage prompt looks like this:
An incident is open on https://status.cloud.microsoft/. Until the status is Service restored or Available, watch it more closely. Check the current Microsoft 365 post and RSS feed at least every five minutes, and again at the "Next update by" time Microsoft has published, in Sydney time.
Only message me when something has changed: the status, the title, the list of affected services, or the current status text. Quote the new current status in a few lines, say what changed since the last update, and give the next promised update time. If that time passes with no new post, tell me once. Do not notify customers. Do not invent progress that Microsoft has not posted.
That is also how I would have wanted it to behave this morning. The public post already had a current status and a previous status in the same message. A bot that only looks at a timestamp can miss a wording change. Compare the text.
Report when it is resolved
When Microsoft marks the issue as restored, I want a short all-clear in the same chat: what was affected, when it started, when it was restored, and a reminder that the public page is a broad signal, not proof that every tenant is back to normal. Then the routine can go back to staying quiet.
The resolution prompt:
If https://status.cloud.microsoft/ marks the current Microsoft 365 incident as Service restored or Available, post a resolution report in this chat and then return to the quieter watch. Include the original title, the services that were listed as affected, when the issue was first posted, when it was marked restored, and a short timeline of the updates you saw. Say clearly that this is the public status, not a tenant-by-tenant confirmation. Do not send any customer message unless I ask you to draft one.
Draft the customer notice, then wait for approval
This is where the same Grok Bot work becomes useful for an MSP. If Exchange Online, Teams, SharePoint or Outlook might be affected, customers will ask. I would rather answer from Microsoft’s own words, in a calm note I have actually read, than type one under pressure.
The important rule is that Grok Bot drafts, and you send. Do not let the routine email clients, post to a Teams channel, or update a status page on its own. Leave the draft in the chat. You decide whether the incident is material enough to mention, whether the wording is right, and when it goes out. The same applies to the all-clear.
A prompt for that draft:
Draft a short customer notice, in plain English, based only on the current public incident at https://status.cloud.microsoft/. Write it as an MSP note I can send if I choose to. Include what Microsoft says may be affected, that Microsoft is working on it, a link to the public status page, and that we will follow up when the status is restored. Keep it factual. Do not claim we have confirmed impact in the customer's tenant. Do not apologise for a fault we did not cause, and do not guess at a resolution time.
Leave the draft in this chat. Do not send it, email it, or post it anywhere. I will approve or edit it first. Also draft a matching all-clear I can use once Microsoft marks the issue restored, and hold that back as well.
If you later want the approved text in Outlook or Teams, you can ask the bot to place a draft where you work. That is a separate, approved step. It is not part of the watch routine.
A deeper option: tenant Service health
The public page is the wide, no-sign-in view. Microsoft’s fuller picture for a given organisation still lives in the Microsoft 365 admin centre, under Health, then Service health, which the same Learn article describes. That view is tenant-specific. It lists incidents and advisories that apply to the services that tenant uses, with an issue ID, a history of updates, and, if you want them, email notifications. Roles such as Service Support admin and Helpdesk admin can see it. Planned maintenance is not in Service health; that sits in Message centre.
The programmable version of that tenant view is the service communications API in Microsoft Graph. The permission for health issues is ServiceHealth.Read.All, as delegated or application access, granted by an administrator. Message centre posts are a different permission. Unattended monitoring is the application-permission path, which Microsoft describes for backend alert services in Access service health and communications in Microsoft Graph.
I already have a Grok Bot doing that deeper check in a tenant security routine. It is not watching the public page. It reads Service health for that tenant and includes it in a morning report. On a typical day it has found a dozen or more unresolved items, mostly advisories – a Copilot agent opening the wrong window, a SharePoint knowledge-agent limitation, an Exchange group-reply problem – rather than a wide outage. That is useful, and it is a different job. The public page would have been quiet on those mornings. The tenant view is noisier and more precise. If you add it, keep the public-page routine as the early, permission-free alert, and treat tenant Service health as the follow-up that answers “is this likely affecting this customer?”
I have also written about using a specialist Grok Bot when something in a tenant already looks wrong, in How Grok Bot Simplifies Security Checks and Investigations. Status monitoring is the earlier step: know that Microsoft has posted an issue before you start digging.
Sample prompts
These are the four prompts together, so you can paste them as you set this up.
Routine setup:
Create a routine called Microsoft 365 public status watch. Every 15 minutes during my Sydney business hours, check https://status.cloud.microsoft/ and https://status.cloud.microsoft/api/feed/mac. Compare the title, status and message with the last run. Stay silent if nothing has changed. If a new incident or service degradation appears, or if the post has changed, alert me in this chat with the title, status, affected services, what changed, the next update time in Sydney time, and a link to the page. Do not invent details. Do not sign in to a tenant. Do not contact customers.
During an outage:
Watch the open incident on https://status.cloud.microsoft/ more closely until it is Service restored or Available. Check at least every five minutes, and at the "Next update by" time. Only message me when the status, title, affected services or current update text changes, or if the promised update time passes with no new post. Do not notify customers.
Resolution report:
When the public Microsoft 365 incident is marked Service restored or Available, post a short resolution report in this chat: original title, affected services, first-posted time, restored time, and a brief timeline. Remind me this is the public status, not a per-tenant confirmation. Then return to the quieter watch. Do not send a customer message unless I ask for a draft.
Customer notice draft:
Draft a short customer notice from the current public incident only. Say what may be affected, that Microsoft is working on it, and link to https://status.cloud.microsoft/. Do not claim confirmed impact in a customer's tenant, and do not guess a resolution time. Leave the draft in this chat. Do not send it. Also prepare an all-clear draft and hold that back until I approve it.
What I want from this
The useful design is a Grok Bot that watches one official public page, speaks when the post changes, stays with the incident, and writes a customer note I still have to approve. It will not replace the admin centre, and it will not tell you with certainty that a given tenant is affected. It will mean you heard about the outage from your own bot, in your own chat, with Microsoft’s wording in front of you, before the first customer call.
I used to think the measure of a good business was what it knew. Its products, its processes, the expertise its people had built over years. Those still matter. But I’m increasingly convinced the real measure of future success is something else entirely: how well you cope when everything you know keeps shifting under your feet.
And here’s the uncomfortable part. Not everyone has the skills to do that.
Change isn’t an event anymore
We used to treat change as a project. A migration, a new system, a restructure. You pushed through it, things settled, and you went back to normal. That rhythm is gone. Microsoft 365 alone changes every week. Copilot picks up new capabilities, agents arrive, settings move, licences get reshuffled. Then there’s everything happening outside your tenant.
This isn’t just my gut feel. The World Economic Forum’s Future of Jobs Report 2025 found that nearly 40% of skills required on the job are set to change, and that 63% of employers see the skills gap as the key barrier to transforming their business. It also notes that human skills like resilience, flexibility and agility will remain critical alongside technical ones.
Read that again. Resilience and flexibility are being named next to AI and cybersecurity. Coping with change is now a core skill.
Why some people thrive and others stall
Picture two people in the same team when Copilot lands in Teams and Outlook. One pokes at it, gets a few things wrong, adjusts and within a month has quietly rebuilt how she runs her week. The other waits for training, waits for someone to tell him the rules, and eventually decides it isn’t for him. Same tools, same opportunity, very different futures.
The difference usually isn’t intelligence. It’s comfort with not knowing. Some people find uncertainty interesting. Others find it threatening. I don’t blame the second group. Many were rewarded for years for knowing the answer, not for working one out.
I wrote recently about this gap in Same Licence, Different Results. The licence is identical. What people bring to it is not.
Owners set the pace
If you own a business or an MSP, this lands on you first. Your team takes its cue from how you handle change. If every new thing is met with a sigh and a “we’ll look at it later”, that becomes the culture. If you visibly try things, admit what didn’t work and share what did, that becomes the culture instead.
A few things I’d focus on:
Make experimenting normal. Give people time to play with Copilot on real work, not just in a training session.
Shrink the cost of being wrong. Small trials in a pilot group beat big bets made once a year.
Teach how to learn, not just what to click. The buttons will move. The habit of figuring things out won’t.
Notice who is struggling. Some people need more support through change, and that’s fine. Leaving them behind isn’t.
For MSPs there’s an opportunity here too. Your customers don’t just need the technology deployed. They need help absorbing a constant stream of change. That’s a service, and a valuable one.
The skill underneath the skills
I don’t think the winners over the next few years will be the people who know the most about AI today. Today’s knowledge has a short shelf life. The winners will be the workers, owners and businesses who can keep re-learning without falling apart each time the ground moves.
That’s a skill. It can be built. But only if we admit it’s missing and start practising it on purpose.
Most MSPs already have a folder of PowerShell scripts that do the real work: connect to a tenant, check a setting, produce a report. The awkward part is not writing the next script. It is getting a consistent, least-privilege way to run the ones you already trust, without giving every assistant in your toolkit its own path into Microsoft 365.
I keep a dedicated PowerShell bot in Grok Bot for this. It is not there to chat about PowerShell in general. It is a tool. When a security monitor, or my Chief of Staff, needs a script run, they ask that bot to do the job and report back. The rest of my bots stay out of the tenant.
That sounds simple, and once it is set up it is. The work is in giving the bot a clear job, pointing it at a script repository, and letting it sign in with a certificate so later runs can go without a prompt. I will walk through that setup, then use the Microsoft Defender for Office 365 Recommended Configuration Analyzer (ORCA) as the worked example.
Why a dedicated PowerShell bot
If every bot can run scripts, every bot needs tenant access, and every bot can make a change. I would rather have one specialist with a tight set of rules: use the scripts in the repo before writing anything from scratch, stay read-only unless I say otherwise, ask before anything that changes a tenant, and report back in plain English.
Other bots can then call it when they need it. That is not a theory. A marketing bot has asked mine for a read-only inventory of a script repo. A chief-of-staff bot has sent it backup and investigation jobs. The PowerShell bot does the work and replies. I wrote earlier about using a specialist bot to investigate a worrying mailbox change. This is the same idea, pointed at scripts.
1. Create the bot and give it a clear role
You can do this in chat. The only app locations I will point you to are ones I have verified: the per-agent info pane, and Settings if you need to check the timezone later.
Click the agent’s name in the chat header, or use Cmd+Shift+I, to open the per-agent info pane. The gear beside the X opens the per-agent settings for avatar, name, title and description. Call it PowerShell, or whatever name you will use when another bot refers to it, and give it a description that states the job.
You can also set the role in chat:
Set this bot’s name to PowerShell and set the description to: You are a specialist PowerShell tool for an Australian MSP. Other bots will ask you to run scripts and report back. Prefer scripts from my GitHub repositories, and pull the latest copy before you run one. Run scripts on your computer. Use certificate-based app-only sign-in for unattended Microsoft 365 work. Stay read-only unless I clearly ask for a change. Before anything that changes a tenant, tell me exactly what you will do and wait for approval in this chat. Do not invent a script if a repo already has one that fits. Report results in plain English, and say what you could not do.
Save the same rules as a standing instruction in the first conversation. Be explicit that relayed approvals from another bot do not count for tenant changes. That is a rule I have had to enforce, and it is the right one.
2. Point it at a GitHub repo
A PowerShell bot without a script library will invent helpers. I would rather it used the scripts I already maintain. Mine lives at https://github.com/directorcia/Office365. Tell the bot that in chat:
Add https://github.com/directorcia/Office365 to your list of script sources. When a job needs a script, check that repo first, pull the latest copy, and run it from there if it fits. Do not write a new script if one in the repo already does the job. Do not commit or push anything unless I ask.
That repo is public, so the bot can clone and pull it on its own computer without a token. If you also have a private repo, the safest approach I have used is a fine-grained GitHub token limited to that one repository, with Contents set to Read-only. GitHub turns on Metadata read-only by itself. Leave every other permission at No access. The token is for clone and pull only, and it should not be stored in the repo.
I am not going to invent a GitHub settings screen in the Grok Bot app. I set this up in chat, then confirmed the bot could see the repo on its computer. Before each run, I expect it to pull the latest copy.
Interactive sign-in is fine for a one-off on your own PC. It is a poor fit for a bot. Nobody is sitting at the bot’s computer to complete a browser prompt, and a prompt that nobody can answer will just hang. Certificate-based, app-only sign-in is the unattended path Microsoft documents for Exchange Online PowerShell.
The pattern I use is:
Generate the certificate on the bot’s computer, so the private key never leaves that machine.
Upload the public certificate to an app registration in Microsoft Entra ID.
Assign a role that matches the work. For a read-only assessment such as ORCA, that is Global Reader or the Exchange View-Only Organization Management role group, not a full administrator role.
The connect script can also provision the Entra app. Read what it will assign before you use that option. A provision step that grants a broad Exchange role is the wrong default for a bot that mostly reads a tenant. I would rather start with a view-only role and add more later, with a human still in charge of the grant.
The first time, you will need to sign in once with a device code so the app can be created and consent can be granted. After that, the bot should connect with the saved profile and the unattended switch. On the Linux script, that is -UseCertificateAuth with -Headless (an alias of -noprompt):
Keep the PFX password out of the repo and out of chat prompts. The script will take it from a secret or from an environment variable you inject. Each command on the bot’s computer can start a fresh PowerShell process, so either reconnect each time or keep a session open per tenant. A different tenant needs its own connection.
4. Let other bots call it
This is the point of the setup. You do not want your news bot, or your security monitor, running Exchange cmdlets itself. You want it to ask the PowerShell bot.
I do not have a separate settings page I can point you to for that. What I can verify is simpler. You ask the other bot, in its own chat, to send the job to the PowerShell bot and report back. In a group chat, the per-agent info pane – the same one you open from the agent name in the header, or with Cmd+Shift+I – includes Members, and you can @mention a bot by name.
Here is the kind of prompt I would give a security monitor or a chief-of-staff bot:
Ask the PowerShell bot to run ORCA read-only against the tenant I name, using the Exchange certificate connection. It should install or update the ORCA module if needed, generate the report, summarise the results in plain English, and send back the HTML report. It must not change anything in the tenant. If it hits a permission gap, it should say so and stop, without granting itself extra access.
The PowerShell bot then does the work and replies. Keep change control in its own chat. If another bot relays “Robert said yes”, that is not the same as you typing the approval where the script will run.
You can also put a recurring check on the calling bot, not on the PowerShell bot. A routine lives in that bot’s per-agent info pane, under the computer preview. Check the timezone under Settings, on the General tab, opened from the sidebar account button at the bottom-left, or with Cmd+,. Routines usually start within a few minutes either side of the time you set. If the run must start right on the minute, say “exactly” or “sharp”. The routine prompt can simply tell the monitor to ask the PowerShell bot for this week’s ORCA report.
5. A worked example: running ORCA
ORCA is a PowerShell module that compares your Exchange Online Protection and Microsoft Defender for Office 365 settings with Microsoft’s recommended Standard and Strict values. Microsoft documents it alongside the recommended threat policy settings: you install the module from the PowerShell Gallery, connect to Exchange Online, and run Get-ORCAReport. It produces an HTML assessment of anti-spam, anti-phishing, Safe Links, Safe Attachments and related controls.
You also need the Exchange Online Management module. To run ORCA, the official project says you need the Exchange Online View-Only Configuration role. In practice that means a read-only role such as Global Reader or View-Only Organization Management, which is exactly the access I already wanted the bot to have.
Microsoft also has a configuration analyzer in the Defender portal. That remains the better long-term check, because the current gallery release of the ORCA module, 2.8.1, was last updated in December 2023. For an MSP, the value of running it from a bot is speed and repeatability, not that it replaces the portal.
Once the PowerShell bot exists, the request is short:
Run https://github.com/cammurray/orca against [tenant] read-only. Use the Exchange certificate connection. Install the ORCA module if it isn’t already there, run Get-ORCAReport, summarise the findings in plain English, and give me the HTML report. Do not change anything in the tenant.
That is close to the prompt I actually used. The bot connected with the certificate, ran the module, and sent the report back. In one tenant it finished in about five minutes, and it only read the tenant. Of 67 checks, 50 were OK, 5 were informational and 12 were recommendations. In another tenant the pattern was similar: 50 OK, 4 informational and 13 recommendations. Typical findings were anti-phish actions left at NoAction, a low phishing threshold, Safe Links allowing click-through or missing coverage on some domains, and outbound spam limits that did not match the recommended values.
Two caveats are worth knowing if the bot’s computer is Linux, which mine is. The DKIM and SPF checks use Resolve-DnsName, which is a Windows cmdlet, so those checks may not run. In one run a Safety Tips check also failed with a server-side error, and a protection alerts check was skipped because it does not work over a certificate sign-in. None of that is a reason to avoid the report. It is a reason to read the gaps as well as the recommendations.
The point is the effort on my side. I did not install modules by hand, hunt for the right connect script, or click through a report wizard. I asked the bot I already had, and it used the repo and the certificate I had already set up.
Sample prompts
These are the prompts I would start with if you are setting up the same pattern.
Create the bot:
Set this bot’s name to PowerShell and set the description to: You are a specialist PowerShell tool for an Australian MSP. Other bots will ask you to run scripts and report back. Prefer scripts from https://github.com/directorcia/Office365, and pull the latest copy before you run one. Run scripts on your computer. Use certificate-based app-only sign-in for unattended Microsoft 365 work. Stay read-only unless I clearly ask for a change. Before anything that changes a tenant, tell me exactly what you will do and wait for approval in this chat. Report results in plain English, and say what you could not do.
Add the repo:
Add https://github.com/directorcia/Office365 to your list of script sources. When a job needs a script, check that repo first, pull the latest copy, and run it from there if it fits. Do not commit or push anything unless I ask.
The prompt another bot sends to call it:
Ask the PowerShell bot to run ORCA read-only against the tenant I name, using the Exchange certificate connection. Summarise the results in plain English and send back the HTML report. Change nothing in the tenant. If you hit a permission gap, say so and stop.
The ORCA run itself:
Run ORCA read-only against [tenant]. Connect with o365-connect-exo-cert-linux.ps1 using the saved profile and -Headless, then run Get-ORCAReport. Summarise the OK, informational and recommendation counts, list the important recommendations, and attach the HTML report. Tell me if any checks could not run, including DNS checks on Linux. Do not change anything in the tenant.
What I took from it
The useful design is not “give every bot PowerShell”. It is one specialist, a repo of scripts you already trust, and a sign-in that can run unattended without handing out a full administrator role.
Keep the human in charge of anything that changes a tenant, and keep that approval in the PowerShell bot’s own chat. Use official documentation when you check what a module is supposed to do – the Microsoft Learn connector is the straightforward way to pull that wording. And start with a real job, not a tour of features. For me, that job was ORCA. Once the bot was in place, running it was a short request, a few minutes of read-only work, and a report I could act on.
For the last couple of years, most AI conversations I’ve had started the same way. Which model is best? Which one is smartest this month? Who just topped the leaderboard? I get why. The models were the shiny part. But I’m noticing the question changing, and I think it’s the right change.
The model is becoming the engine. What actually decides whether AI is useful in your business is everything bolted around it: the harness, the tools, the data it can reach and the rules it works within.
The engine isn’t the car
A brilliant model sitting on its own knows a lot about the world and nothing about your business. It can’t see your customer list, your contracts or yesterday’s Teams meeting. It can’t raise a ticket, update a Planner task or check a device in Intune. Without connections, it’s a very well-read stranger.
Microsoft 365 Copilot is a good example of where the value really sits. Microsoft describes Copilot experiences as differing by data grounding, integration depth and licensing, with work grounding coming through Microsoft Graph. Notice what that list doesn’t lead with. It’s not the model. It’s what the model is plugged into.
Picture two businesses using the same underlying model. One just chats with it. The other has it grounded in SharePoint, connected to their line-of-business system and able to run a Power Automate flow. Same brain, wildly different outcomes.
Microsoft has already brought it into Copilot Studio, where you can extend an agent with tools using MCP. That changes how I think about building agents. The hard work is no longer picking a model. It’s deciding which tools an agent gets, what data it can see, and what it’s allowed to do with them.
I touched on a version of this when I wrote Why developers don’t own this one. The value of an AI tool at the command line came from what it could reach and run, not from the model’s raw IQ.
What this means for MSPs
If you run an MSP, this is good news. Model choice is turning into a commodity decision that vendors will keep changing underneath you. The harness is where your expertise lives. That’s the bit you can design, secure and support.
Here’s where I’d focus:
Grounding: is the data Copilot reaches current, permissioned properly and worth trusting?
Connectors: which systems should an agent touch, and which should it never see?
Guardrails: what can an agent read versus actually change, and who approves the risky steps?
Identity and audit: can you tell, through Entra and Purview, what an agent did and on whose behalf?
That last point matters. Every new tool you connect is also a new door. Exposed or poorly authenticated MCP servers have already shown up in security research I covered in the CIA Brief 20260516. A great harness is a secure harness.
Stop asking which model
I’m not saying models don’t matter. Better reasoning helps. But the gap between good models is shrinking, while the gap between a well-connected agent and a lonely chatbot is enormous.
So next time someone asks me which AI is best, I’ll answer with a question. Best at what, connected to what, and trusted to do what? Get those answers right and the model almost picks itself.
A Full Access grant on a mailbox is the kind of audit event that makes you sit up. Someone, or something, has just been given the right to open a mailbox and read what is in it. In an MSP or SMB tenant that is not a change you ignore. It is also, as I was reminded this week, the kind of change Microsoft’s own Exchange service can generate as routine maintenance. The hard part is telling those two things apart quickly, without handing extra privilege to the tool that is helping you look.
My read-only Microsoft 365 monitoring bot flagged a burst of mailbox changes at 6:02–6:03pm the previous evening, including a Full Access grant. It did not assume the worst, and it did not start changing anything in the tenant. It asked me a straight question: was that burst me, or a Copilot or agent test?
I had not made any mailbox changes. I told it to investigate further. What follows is how that investigation ran, what the issue turned out to be, and how you can set up a similar read-only check of your own.
Why audit noise is hard to work with
The Microsoft Purview unified audit log is one of the most useful tools in a Microsoft 365 tenant. It is also one of the noisiest. A genuine mailbox permission change sits in the same log as scheduled Microsoft maintenance, directory sync, and admin work you already know about. The record is often technically correct and still hard to read at a glance.
That problem is sharper for MSPs and smaller IT teams. An MSP is watching this pattern across many tenants, usually without a 24-hour security operations centre. An SMB often has one or two people who also run the rest of the environment. In both cases the failure modes are the same. You either spend the evening chasing a change that Microsoft already documented as normal, or you scroll past something that deserved a closer look.
Alerts do not solve this by themselves. An alert can tell you that a Full Access grant happened. It cannot tell you whether the actor was a person, an app you own, or Exchange doing maintenance. It also cannot tell you, in plain English, what was touched and what was not.
I already use Grok Bot for a morning brief of MSP news and security updates. That is a different job. Industry news will not investigate a change inside your tenant. For that, you need a bot with read-only access, a habit of asking you first, and a willingness to stop when it cannot see enough.
How Grok Bot helps
The value is not that the bot is cleverer than the audit log. The log already has the events. The value is the way the work is done:
It monitors in read-only mode, so a routine check cannot become a change in the tenant.
It asks the human first when something looks unusual, instead of guessing or firing a ticket.
It investigates on request, then reports findings in plain English.
It states gaps honestly. If it cannot confirm a permission is still in place, it says so, rather than quietly requesting more access for itself.
When more access is genuinely needed, the human grants it, after the bot has explained what it needs and what the trade-offs are.
It can pause for a safety check, including whether you are looking at the right tenant.
That last point matters more than it sounds. A useful investigation bot is sceptical, not eager.
A real investigation
The bot’s first message was the right one. It had seen a burst of mailbox changes at 6:02–6:03pm, including a Full Access grant, and it wanted to know whether that was me or a test.
My monitoring bot asked whether the mailbox change burst was me, then investigated read-only when I said it wasn’t.
I replied that I had made no mailbox changes the previous day, and asked it to investigate further.
It stayed read-only. The actor was not a user, and it was not one of my apps. It was Exchange’s built-in service account, NT SERVICE\MSExchangeAdminApiNetCore, running from inside Microsoft’s own servers. It had only touched Exchange’s built-in system mailboxes. No user mailbox was changed, and no forwarding was set.
The Full Access grant was also narrower than it first appeared. It gave the built-in Discovery Management role group access to the Discovery Search mailbox. That is Exchange’s eDiscovery role group, and Microsoft documents this permission as the default. The same pattern had already run about 20 times between 12 October and the day of the alert. It only looked suspicious because the audit record showed the account’s identifier rather than its name.
That reading matches Microsoft’s own documentation. In the audit log activities guidance, Microsoft says you may see NT SERVICE\MSExchangeAdminApiNetCore on mailbox permission events, that this is a system account in the Exchange service in the Microsoft cloud, and that a common audited activity is updating permissions on the Discovery Search mailbox to verify that Full Access — “which is the default” — is assigned to Discovery Management. If you want to check that kind of claim while you investigate, the Microsoft Learn connector in Grok Bot is the straightforward way to pull the official wording rather than relying on memory.
So the story in the audit log was already pointing to routine maintenance. There was still a gap.
Closing the last gap
The bot was honest about what it could not do. It could see the audit events, but the read-only app could not read current Exchange mailbox permissions, so it could not confirm the Full Access grant was still in place. It did not request extra permission for itself, and it did not grant itself any.
Instead, it explained the access it needed, and I granted it. That was the Exchange.ManageAsApp application permission, with admin consent, plus the Global Reader role.
This is where least privilege deserves a proper look. Exchange.ManageAsApp lets an app connect to Exchange Online PowerShell without a user sign-in, but it is not a blank cheque. Microsoft still requires an Entra role or an Exchange role group to decide what the app is allowed to do. Global Reader is a supported choice for that, and Microsoft describes it as the read-only counterpart to Global Administrator: it can read settings and administrative information across Microsoft 365 services, but it cannot take management actions.
It is still broader than this particular check required. In Exchange Online, Global Reader maps to the View-Only Organization Management role group, whose members can view the properties of any object in the organisation. If all you need is Exchange visibility, adding the app to View-Only Organization Management is the narrower path. Global Reader also grants read access across other Microsoft 365 admin centres. The bot pointed that alternative out. I chose Global Reader anyway, because I wanted the broader read-only view for this monitoring bot and I was the person granting it. The trade-off was explicit. That is the standard I want: the human decides, with the narrower option on the table.
There was one more useful pause. I sent a screenshot from the admin portal so the bot could see the consent and role grant. The header showed a name that made it stop and warn me I might be in the wrong tenant. It was actually this tenant’s display name. I would rather a bot hesitate over a familiar name than charge ahead in the wrong directory.
With the access in place, the bot connected app-only to Exchange Online PowerShell, ran only Get-* read cmdlets, and disconnected. It confirmed that Discovery Management still has Full Access on the Discovery Search mailbox. The permission is not inherited, and it is not a deny entry. That matches Microsoft’s documented default, and it was still in place after the maintenance. Nothing in the tenant was changed. The evidence was written to an investigation file with no secrets in it.
The outcome was straightforward: routine Exchange Online maintenance, no incident. It was all resolved the next morning, within minutes of the access being granted.
Example prompts
These are the kinds of prompts I would start with if you want the same behaviour.
Set the bot’s role:
You are a read-only Microsoft 365 security monitoring bot for an MSP. Watch the unified audit log for unusual mailbox permission changes, forwarding, and similar high-impact events. When you see something that looks unusual, ask me whether it was expected before you investigate. Stay read-only. If you hit a permission gap, tell me what you cannot see and what access would close that gap. Do not request or grant yourself extra permission.
Create a recurring check:
Create a routine called Mailbox permission watch. Every weekday at 8:15am Sydney time, search the unified audit log for mailbox permission changes since the last run, including Full Access grants and forwarding changes. If nothing material happened, tell me that in this chat and stop. If something looks unusual, summarise it in plain English and ask me whether it was expected before investigating further. Stay read-only.
When an alert appears:
I made no mailbox changes yesterday. Please investigate further. Stay read-only. Identify the actor, what was changed, whether any user mailbox was touched, and whether forwarding was set. If the actor looks like a Microsoft system account, check the official Microsoft documentation and tell me if this is a known maintenance pattern. If you cannot confirm the current permissions, say so clearly and do not try to expand your own access.
After you have granted extra read access:
I have granted the access we discussed. Connect app-only to Exchange Online PowerShell, run only Get-* cmdlets, confirm whether the permission is still in place, then disconnect. Record the findings in an investigation file with no secrets. Change nothing in the tenant.
How to set up a similar read-only check
You can do most of this in chat. The only app locations I will point you to are ones I have verified: the per-agent info pane, and Settings for timezone.
1. Give a bot a clear role
Use a dedicated bot for tenant monitoring, so investigation work is not mixed with unrelated chats. Click the agent’s name in the chat header, or use Cmd+Shift+I, to open the per-agent info pane. The gear beside the X opens the per-agent settings for avatar, name, title and description. Give it a clear description: read-only monitoring, ask first, investigate on request, state gaps honestly, never escalate its own permissions.
2. Start with read-only audit access
Give that bot read-only access to the unified audit log through an app you control. Start with enough access to search the log, and nothing more. The investigation above got a long way on audit events alone.
3. Save the behaviour in chat
Use the role prompt above. Be explicit that the bot must ask you before digging in, must stay read-only, and must not request or grant itself extra permission.
Create the routine in chat. You can review saved routines in the same per-agent info pane, under the computer preview. Check the timezone under Settings, on the General tab, opened from the sidebar account button at the bottom-left, or with Cmd+,. Routines usually start within a few minutes either side of the time you set. If the run must start right on the minute, say “exactly” or “sharp”.
6. Grant extra access yourself when a gap appears
For a current mailbox permission check, that means Exchange.ManageAsApp with admin consent, plus a view-only role. Prefer View-Only Organization Management if you only need Exchange visibility. Use Global Reader only if you have a reason for the broader Microsoft 365 read access, and make that choice deliberately.
7. Keep an investigation trail
A file of findings with no secrets in it is more useful the next time the same maintenance burst appears.
What I took from it
The first lesson is the oldest one in this kind of work: verify before you panic. A Full Access grant is worth taking seriously. In this case it was Microsoft putting a documented default back where it already belonged. The audit log looked worrying because it showed an identifier instead of a familiar name. That is not a reason to ignore the alert. It is a reason to investigate it properly.
The second is least privilege. The bot did not help itself to more access when it ran out of visibility. I granted the extra permission, and I accepted a broader role than the narrowest option on the table. I am comfortable with that choice. I would not have been comfortable if the tool had made it for me.
The third is to keep the human in charge. The useful pattern was simple. The bot watched, asked, investigated, admitted a gap, explained the access it needed, paused when a screenshot might have been the wrong tenant, ran only read commands, and wrote down what it found. I still made the calls that mattered.
That is what simplified the security check. Not a dashboard, and not a promise that every alert is harmless. I went from a worrying Full Access grant to a documented, no-incident conclusion the next morning, without the tool quietly expanding its own reach. For an MSP, that is the difference between an audit event that ruins the start of the day and one that has already been explained.
Most businesses I work with have the same retention policy, even if nobody wrote it down: keep everything, forever, just in case. Storage feels cheap, deleting feels risky, and nobody wants to be the person who threw out the one file the lawyers asked for.
I get it. But in a world of Copilot and constant breaches, “keep everything” isn’t the safe option anymore. It’s quietly making both your security and your AI worse.
More data, more risk
Every file you keep is a file someone can steal. An old client spreadsheet from 2014 does nothing for you today, but it’s just as damaging when it turns up in a breach. The bigger the pile, the bigger the attack surface and the more there is to lose.
Old data is also usually the worst-governed data. Sites from long-finished projects still have “Everyone” links. Ex-staff’s shares never got cleaned up. And if you ever end up in a dispute, everything you kept is potentially discoverable, including things you’d forgotten existed.
For Australian businesses there’s a legal angle too. Under APP 11.2, once you no longer need personal information for any permitted purpose, you must take reasonable steps to destroy or de-identify it (with limited exceptions). Hoarding isn’t neutral. It can be a compliance problem.
Why Copilot cares about your mess
Microsoft is clear that Copilot “only surfaces organizational data to which individual users have at least view permissions.” (Microsoft Learn). That’s good security design, but it means Copilot grounds its answers in everything a user can reach, rubbish included.
Picture asking Copilot for the current pricing policy. It finds five versions across SharePoint, Teams and someone’s OneDrive, including the 2019 draft that was never approved. Stale, duplicated and conflicting content produces confused answers, and users then decide “AI doesn’t work”.
Worse is oversharing. Copilot doesn’t create permission problems, it reveals them. The salary file that was technically open to everyone but buried six folders deep can suddenly appear in an answer. I wrote more about this in Data Governance Is the Real Copilot Readiness Test.
What good looks like: Microsoft’s tools for tidying up
The good news is Microsoft 365 already has the tools. Here’s where I’d start:
Buy time while you clean up.Restricted Content Discovery limits specific sites from appearing in organisation-wide search and Copilot responses while you review them.
Deleting things feels scary. Keeping everything feels safe. The reality is the reverse. A smaller, well-governed data estate is easier to secure, cheaper to defend in a dispute, and gives Copilot something worth reasoning over.
If you’re an MSP, this is a conversation worth having with every client now. Start with retain-then-delete and inactive sites. Your clients’ security and their AI results will both thank you.
Microsoft is retiring the SMS and voice MFA it delivers itself, and I’ve already seen plenty of confusion about what actually happens and when. Message Center post MC1519855 has a few dates in it, and they’re easy to squash together. So here’s how they fit, quoted straight from Microsoft’s documentation.
The short version: Microsoft stops sending the texts and making the calls, but you can keep using SMS and voice MFA after that by bringing your own telephony provider. Nothing has to be done by 30 October or 1 November. That said, the clock is definitely ticking.
What each date means
30 October 2026: the third-party provider option becomes available. Nothing is turned off. The Message Center post says: “Starting October 30, 2026, Microsoft Entra ID will make Choose Your Own Telephony Provider (CYOT) generally available.” and “organizations can choose to configure a supported telephony provider.” It’s something you opt into, not a deadline. Microsoft-provided SMS and voice keep working as they do now.
1 February 2027: Microsoft stops providing SMS and voice for most users.Microsoft Learn says: “From February 1, 2027, Microsoft-provided telephony delivery for SMS and voice will be retired for all users except Global Administrators and external users.” Internal guest users are included in this date.
1 July 2027: the same retirement applies to Global Administrators and external users.
Already happening, since 1 September 2026: users enabled for SMS or voice “are auto-enabled and nudged for Passkey registration upon MFA sign-in.” That’s why your users may already be seeing passkey prompts.
One wrinkle worth knowing. The Message Center says the provider option is generally available from 30 October, but at the time of writing the Learn page on telephony providers still describes it as a private preview. I expect the documentation to catch up, but don’t be surprised if what you read on Learn and in the admin centre don’t quite line up for a while.
What users see after their retirement date
If a user’s only MFA method is SMS or voice and the tenant hasn’t set up a provider, they’re “required to register a passkey during sign-in to continue accessing their account. This prompt is blocking.” Microsoft adds: “There is no opt out.”
Picture the Monday morning helpdesk queue at a 30-seat business where half the staff never moved off text codes. Users who already have Authenticator, a passkey, Windows Hello or another method aren’t blocked, although they may still be nudged to register a passkey. Everyone else hits a wall.
Keeping SMS and voice after February
Yes, you can. Microsoft Learn says the option “lets organizations continue using SMS or voice authentication with a supported telephony provider.” Here’s what that involves:
Where you buy it: you choose the provider through Microsoft Security Store, and “manage their commercial relationship directly with the provider.” The provider bills you, not Microsoft.
Which providers:“Soprano and Telesign are the initial telephony providers” in the private preview, with “more providers” by general availability. Soprano offers per-user or per-transaction pricing.
What you need to buy it:“an Azure billing subscription and resource group, plus the appropriate Azure subscription role.” Microsoft also says to review “provider pricing and expected Azure infrastructure charges,” and links a cost calculator, so expect some Azure hosting cost on top of the provider’s fees.
Who does what: Entra “continues to enforce your authentication method policies,” and the provider just delivers the texts and calls. You’re responsible for “reviewing its regional coverage and terms, completing the provider agreement, configuring the provider… and monitoring the service.”
Microsoft’s own advice: use a provider “only for user populations that have a business, regulatory, or technical requirement for telephony-based authentication.”
I couldn’t find Australian pricing or confirmation of Australian coverage in what’s been published so far, so check those with the provider directly before you promise anything to a client.
What MSPs should be doing now
Find out who uses SMS or voice in every tenant. Microsoft provides a PowerShell script for this, which needs Global Reader, Authentication Policy Administrator or Security Reader.
Move those users to Authenticator or passkeys well before 1 February, and get Global Admins done before 1 July. Admins shouldn’t be on SMS anyway.
Tighten the policies behind it. Once users are off text codes, make sure Conditional Access actually requires the stronger methods. I covered why that matters in Your Conditional Access Is Stuck in 2018.
Only consider a provider for the genuine edge cases, such as shared phones, sites where phones aren’t allowed, or a regulatory requirement, and budget for the provider’s fees plus the Azure costs.
Like everything else in security, this is a shared job. Microsoft is giving us passkey defaults and a provider option. It’s up to us to get users across before the dates arrive, rather than letting a blocking prompt do it for us.
MC1519855 in the Microsoft 365 admin centre Message Center (requires sign-in). Public copy: mc.merill.net/message/MC1519855, which is a community archive, not a Microsoft site.
