Tracking failed logins using Cloud App Security

Previously, I have said that Office365/Microsoft Cloud App Security is

A great security add on for Microsoft 365

I’ve also detailed the differences in the plans here:

Cloud App Discovery/Security

What many people want to know more information about is failed logins to their tenant, so Cloud App Security to the rescue!

Start by navigating to the Cloud App Security Activity log as shown above. Then select the Advanced option in the top right as shown.

Doing so should reveal the ability to define filters as show above.

I’d also recommend that you go and define a “safe” range of IP addresses like I have detailed here:

Define an IP range in Cloud App Security

In my case I have defined some known safe IP’s as “Corporate”.

So the first line of my query basically excludes any of these known IP addresses in the results. That is, I’m looking for failed logins outside my corporate environment. This will generally exclude average users failing to login to their accounts inside my environment, which happens a lot. The idea with this is simply to reduce the noise of lots of alerts, but if you want to know about all failed logins to your tenant, just exclude this condition.

The second part of the filter is to show activities that equal ‘Failed log on’ as shown. You can customise this further if you want to make it more granular, but for now let’s track any failed login to the tenant.

So the final query should look like the above.

You should be not surprised to see the number of results you get as shown above. In my case, there are failed logins from the US, China, Russia, Italy, etc.

Now of course, I can drill into each item for more details but what I really want is a way for me to be alerted about these when they happen. Cloud App Security to the rescue again.

At the top of the results you should find a button as shown that says New policy from search, which you should select.

You should now see a page like shown above where you can define your policy. You’ll need to give it a name and description. You may want to increase the severity or change the category to suit. You can also select between single or repeated activity.

Now if you want an email alert then you’ll need to select the option Send alert as email and put in your email address as shown above. You may also want to change the Daily alert limit to suit your needs.

When you have completed the configuration, scroll to the bottom of the page and select the Create button.

You should now see that policy in the list in your tenant as shown above.

Cloud App Security is a really powerful tool that I believe is a must have for every Microsoft 365 tenant, because not only can you create your own custom queries but you can also convert those into alerts as I have shown.