Microsoft Secure Score should be your security benchmark

Security is tough. There are many different settings in many different places I know, however my suggestion is that you should start, and continue to use, Microsoft Secure Score as your security benchmark when it comes to the protection of your environment will make things much easier and provide a simple starting point.

To start, visit:

https://securescore.office.com/

You’ll need to login with a Microsoft 365 administration account to view the results.

image

You should then pretty much see your Secure Score, out of 100, front and centre as shown above. Think of this score as an aggregation of your entire Microsoft 365 environment.

To me, your Secure Score should be at least 80% and higher if possible. If it’s not, then you have some work to do.

If your Secure Score is less than 80% and you are not the person responsible for configuring your Microsoft 365 environment then you need to open a dialog with them about improving your score. If you are paying an external business to manage your Microsoft 365 environment then you should ask them to show you what their own  Secure Score is.

– If their Secure Score is LOWER than what your is, then I would suggest it is time to find someone else who is actually serious about security.

– If their Secure Score is EQUAL to what yours is, ask them to show you a plan for how they plan to get your Secure Score to at least 80%. If they are unable to, again, think about whether you should be using them.

– If their Secure Score is HIGHER than yours is, ask them why that is so and how long will it take for your score to equal or exceed theirs.

A well configured tenant, to best practices, will normally come in with a Secure Score of 65% or so. To me, getting a tenant to 80% does require some work but it isn’t all that hard. Remember, good security means expending some effort. This means that if your Secure Score is well below the 65% mark, then you should be taking immediate action to improve it and implement things to best practices as soon as possible.

image

Now go back to your Secure Score console and select the Include menu in the top right as shown and select the Achievable score as shown. This now shows you what Secure Score you could achieve if you implemented everything you are currently paying for (i.e. licensed for). In essence, this shows you how much security stuff you are paying for that has not been enabled. If that is large, then add that item to your security To-Do list as well.

So in summary, in my opinion,

– Anything below a Secure Score of 30% means you are highly vulnerable I believe.

– Anything below a Secure Score of 50% indicates that best practices have not been fully applied.

– Around 67% is the Secure Score you should expect for a tenant configured to best practices and with all security features enabled.

– Around 80% is the Secure score you should be aiming to get to as soon as possible, mindful of the fact that it will required additional configurations to get to this level.

– A Secure Score of 100% should be your ultimate goal over time. Perhaps a better approach is to always be looking to improve your score above the recommended 80% I indicated. This will require many fiddly and time consuming settings throughout your environment BUT remember, each time you complete one of these your environment will be more secure and that fact should also be reflected in your Microsoft Secure Score.

Azure Cloud Shell now available in Microsoft 365

image

If you take a close look at your Microsoft 365 admin center, as shown above, you might see a new icon in the top right. The Azure Cloud shell is now available right from here.

image

If you select that, you’ll then see a PowerShell style window appear at the bottom of page as shown above. Here you can run all your favourite scripts directly in a browser!

I’ve covered the Azure Cloud Shell in previous articles:

Azure Cloud Shell

Connecting to Exchange Online with Azure Cloud Shell

and you can read the Microsoft documentation here:

Overview of Azure Cloud Shell

The only limitation seems to me is that you need an active Azure subscription tied to your Microsoft 365 environment because Azure Cloud Shell does need some storage to operate. But who doesn’t have an Azure subscription in their tenant these days right?

Deploy Office 365 and Azure together

(Hint, this is another reason to ALWAYS sell an Azure subscription when you sell Microsoft 365 if you are a reseller).

Hopefully, Microsoft might allow some included storage in the future for those without an Azure subscription.

Having the ability to run PowerShell directly from the browser with Microsoft 365 is a super handy addition and hopefully the functionality will keep extending with this.

Email filtering reports

image

There are some real nice and helpful email report in your Microsoft 365 Security console if you haven’t taken a look recently. You can pull them up by visiting:

https://security.microsoft.com/securityreports

as shown above. Then selecting Email & collaboration reports on the right.

image

The one I really like is the Mailflow status summary which you can drill into further by clicking on the heading or selecting the View details button.

image

If you then select the Funnel option across the top as shown, you get an idea of the number of bad emails that are being caught by each stage of the filtering process, from top (total in) to bottom (remaining out).

SNAGHTMLc4b85a0

However, the report I love is the one you get when you select the Tech view as shown above. Why? Because this one even shows you results from DMARC as highlighted.

image

Many also allow you to Create schedule as shown above,

image

that allows you to email the reports regularly.

Keep an eye on the reporting areas of your tenant, as they are rapidly improving and expanding!

Exchange Online AV engines

image

I have found that many don’t appreciate that Exchange Online uses anti virus engines from multiple providers, apart from Microsoft.

“We have partnerships with multiple anti-malware technology providers, so messages are scanned with the Microsoft anti-malware engines, two added signature based engines, plus URL and file reputation scans from multiple sources. Our partners are subject to change, but EOP always uses anti-malware protection from multiple partners. You can’t choose one anti-malware engine over another.:

per – How many anti-malware partners do you have? Can I choose which malware engines we use?

So email will be scanned by three (3) engines in total. One from Microsoft and another two from third parties.

What the online world can learn from recent on prem Exchange Server challenges

It has been a pretty challenging few days for those that still manage and maintain on premises Exchange servers thanks to:

HAFNIUM targeting Exchange Servers with 0-day exploits

Throughout which I’ve seen a lot of smug cloud administrators wondering why people still bother with on premises. I think a better use of their energies would be to look at the current situation and learn from it rather than allocating it to self righteousness.

The cloud is a shared responsibility model. This means that both Microsoft and end user now responsible for the security of cloud infrastructure. Luckily, these recent Exchange issues have largely fallen to Microsoft when it comes to the cloud. Where there is room to learn for the rest of us, is in the response to the situation from those battling to contain it.

From everything I have seen online in regards to the HAFNIUM issue, what I find most interesting is the lack of a response plan. Technically, administrators can follow directions, run scripts, patch systems pretty well. However, most seem totally unprepared for this kind of situation, especially at scale. That’s what worries me the most. Why? Because challenges in the cloud can easily be of the same scale and impact.

There have been plenty of examples when services like Azure AD or Exchange Online have been unavailable, but when they have, I’ve seen the same level of, dare I say, panic. Because systems work 99.99% or more of the time ‘on average’, a large amount of complacency begins to creep into the system, especially those charged with maintaining these systems. Thoughts of disaster recovery and outage impact get put on the back burner and never really addressed because there are always ‘higher’ priorities.

What worries me is the dependency we have built into our modern lives, business and economy, to the point where most cannot function if their phones run out of charge. What worries me when I look at the response I see to broad security challenges in IT is simply the lack of a credible contingency plan. A check list of what to do, if you like. Of course, you can’t have a plan for every contingency but some semblance of a plan is better than no plan at all surely?

In the end it comes down to risk analysis. When the sun is shining, risk analysis is the furthest thing from people minds. This however, is exactly the time that it should be a priority because developing a strategy in midst of a crisis does not generally lead to the best outcome. You want to have a checklist of what to do, well in advance of whenever you may need it.

Even though the systems I work with are cloud based are immune from the HAFNIUM (it appears at least), that doesn’t stop me learning from how the unfortunate are dealing with it. I’m watching, learning and preparing, because as the saying goes, “When did Noah build the Ark?”

Before it rained.

Before it rained.

Email overrides are not best practice

I see a lot of email configurations in Microsoft 365 that use some form of override to ‘get around’ a delivery issue. Doing so is simply not best practice and in fact opens you up for additional attacks.

For more information, let’s review the Microsoft document:

Create safe sender lists in EOP

which says:

  • We don’t recommend managing false positives by using safe sender lists, because exceptions to spam filtering can open your organization to spoofing and other attacks.
  • Use Outlook safe senders – This method creates a high risk of attackers successfully delivering email to the Inbox that would otherwise be filtered; however, the user’s Safe Senders or Safe Domains lists don’t prevent malware or high confidence phishing messages from being filtered.
  • Use the IP allow lists – Without additional verification like mail flow rules, email from sources in the IP Allow List skips spam filtering and sender authentication (SPF, DKIM, DMARC) checks. This creates a high risk of attackers successfully delivering email to the Inbox that would otherwise be filtered; however, the IP Allow List doesn’t prevent malware or high confidence phishing messages from being filtered.
  • Use allowed sender lists or allowed domain lists – This method creates a high risk of attackers successfully delivering email to the Inbox that would otherwise be filtered; however, the allowed senders or allowed domains lists don’t prevent malware or high confidence phishing messages from being filtered. Do not use domains you own (also known as accepted domains) or popular domains (for example, microsoft.com) in allowed domain lists.

In short, if you are using white lists or the like you are creating a vulnerability in your environment that attackers can exploit. All inbound messages should be filtered through appropriately configured mail filtering policies. If you want information on setting these appropriately see:

Mail flow best practices for Exchange Online and Office 365

Best practices for configuring standalone EOP

Recommended settings for EOP and Defender for Office 365 security

To get an overall picture of all the message overrides in your environment visit the Security and Compliance admin portal:

image

Locate the Reports option on the left and then select Dashboard as shown, from the expanded options. Then on the right locate the Threat protection status tile as shown and select it.

image

From the pull down options in the top right, as shown above, select Message override.

image

You should now see a nice summary of any messages passing through your environment that are overriding your configurations. Don’t forget that you can also View details table and select to Filter in the top right of this report.

A direct link to this report can be found here:

Threat Protection status – Message override

Overriding policies conditions is something that should be avoided as much as possible, simply because it increases the risk in your environment. Also, if you haven’t already, go take a look at what messages are overriding in your environment today and try to eliminate these to improve your security.

End to End email protection with Microsoft 365–Part 6

This is part of a series of articles about email security in Microsoft 365. Please check out previous articles here:

End to End email protection with Microsoft 365 – Part 1

End to End email protection with Microsoft 365 – Part 2

End to End email protection with Microsoft 365 – Part 3

End to End email protection with Microsoft 365 – Part 4

End to End email protection with Microsoft 365 – Part 5

These articles are based on a model I have previously created, which you can read about here:

CIAOPS Cyber protection model

designed to help better explain expansive security included with Microsoft 365.


Email reporting and auditing

It’s now time to look at all the logging that occurs during even the simply process of receiving and viewing an email. For starters there is:

Message tracing

and

Message trace in the modern Exchange admin center

Message trace in the Security & Compliance Center follows email messages as they travel through your Exchange Online organization. You can determine if a message was received, rejected, deferred, or delivered by the service. It also shows what actions were taken on the message before it reached its final status.

There is also reporting options like:

Mail flow insights in the Security & Compliance Center

and

Mail flow reports in the Reports dashboard in Security & Compliance Center

as well as:

Microsoft 365 Reports in the admin center – Email activity

If you want to specifically look at email security there is:

Email security reports in the Security & Compliance Center

as well as:

Defender for Office 365 reports in the Reports dashboard in the Security & Compliance Center

and

Reports for data loss prevention (DLP)

I have also spoken about the importance of the Unified Audit Logs (UAL) in Microsoft 365:

Enable activity auditing in Office 365

Unified Audit Logs in Microsoft 365

and you need to ensure that these have been enabled so that you can:

View mailbox auditing

Starting in January 2019, Microsoft is turning on mailbox audit logging by default for all organizations. This means that certain actions performed by mailbox owners, delegates, and admins are automatically logged, and the corresponding mailbox audit records will be available when you search for them in the mailbox audit log.

Here are some benefits of mailbox auditing on by default:

  • Auditing is automatically enabled when you create a new mailbox. You don’t need to manually enable it for new users.

  • You don’t need to manage the mailbox actions that are audited. A predefined set of mailbox actions are audited by default for each logon type (Admin, Delegate, and Owner).

  • When Microsoft releases a new mailbox action, the action might be automatically added to the list of mailbox actions that are audited by default (subject to the user having the appropriate license). This means you don’t need to monitor add new actions on mailboxes.

  • You have a consistent mailbox auditing policy across your organization (because you’re auditing the same actions for all mailboxes).

With this auditing enabled you can do things like:

Reporting mailbox logins

and

Search the Office 365 activity log for failed logins

as well as

Audit Office 365 user logins via PowerShell

Many of the reports that you find in the Microsoft 365 Admin area can be scheduled to be sent via email per:

Scheduling compliance reports

Apart from auditing and security you can also do more typical things like:

Viewing mailbox usage

Viewing Email apps usage

The availability of all this data is covered here:

Reporting and message trace data availability and latency

typically being 90 days.


User reporting and auditing

For information more specifically about user logins into the service and the Identity container, the best place to look is in Azure Active Directory (AD).

What are Azure Active Directory reports?

Find activity reports in the Azure portal

Azure Active Directory sign-in activity reports – preview

Audit activity reports in the Azure Active Directory portal

and if you want use PowerShell

Azure AD PowerShell cmdlets for reporting

Device reporting and auditing

There are lots of options when it comes to monitoring and reporting on devices. Apart from what is offered locally you also have:

Intune report

Create diagnostic settings to send platform logs and metrics to different destinations

Manage devices with endpoint security in Microsoft Intune

You can even get telemetry data and analytics reports from your desktop applications via:

Windows Desktop Application Program


Aggregated data reporting and monitoring

As you can see with all the options above, it is easy to get to information overload trying to keep up with all those signals. Luckily Microsoft provides a range of services to aggregate all this for you to make monitoring and report easier.

The first is Microsoft Cloud App Security services:

Cloud App Discovery/Security

Microsoft Cloud App Security overview

Microsoft Cloud App Security data security and privacy

There are plenty of reasons why you really should have Microsoft Cloud App Security in your environment:

A great security add on for Microsoft 365

Office 365 Cloud App Discovery

Next, is Microsoft Defender for Endpoint that will aggregate security and threat information for devices in your environment and make it available in a single console.

Overview of Microsoft Defender Security Center

Microsoft Defender Security Center portal overview

Microsoft Defender for Endpoint

Microsoft Defender for Endpoint evaluation lab

Finally for me, there is Azure Sentinel, which I see as really the ultimate hub for event reporting, monitoring and alrtign across the whole service.

Another great security add on for Microsoft 365

Introduction to Azure Sentinel

Azure Sentinel is a service that growing in features rapidly:

A couple of new additions to Azure Sentinel

Stay ahead of threats with new innovations from Azure Sentinel


Summary

Hopefully, all this gives you some insight into all the auditing and usage data that Microsoft 365 captures during any interaction within the service. One of the biggest benefits is also how this information is integrated between services, especially those that aggregate information lime Microsoft Cloud App Security and Azure Sentinel. This means you don’t have to crawl through individual log entries, you can use a dashboard and drill down from there. I also like the fact that all of these services and data are accessible using a scripting tool like PowerShell if you want to automate this further.

Remember, throughout this six part series I’ve just looked at what happens when a single email is delivered and view with Microsoft 365. If you expand that out to all the services and capabilities that Microsoft 365 provides you can hopefully get a better appreciate of the protection it provides in place for your data on many different levels.

The call to action for readers is to go away and implement all the security features that Microsoft 365 provides. This may of course vary by the license that you have. You should then consider what additional security offerings the Microsoft cloud stack can offer that makes sense for your business, then implement those. Remember, security is not a destination, it is journey.