Tracking failed logins using Cloud App Security

Previously, I have said that Office365/Microsoft Cloud App Security is

A great security add on for Microsoft 365

I’ve also detailed the differences in the plans here:

Cloud App Discovery/Security

What many people want to know more information about is failed logins to their tenant, so Cloud App Security to the rescue!

image

Start by navigating to the Cloud App Security Activity log as shown above. Then select the Advanced option in the top right as shown.

image

Doing so should reveal the ability to define filters as show above.

I’d also recommend that you go and define a “safe” range of IP addresses like I have detailed here:

Define an IP range in Cloud App Security

In my case I have defined some known safe IP’s as “Corporate”.

image

So the first line of my query basically excludes any of these known IP addresses in the results. That is, I’m looking for failed logins outside my corporate environment. This will generally exclude average users failing to login to their accounts inside my environment, which happens a lot. The idea with this is simply to reduce the noise of lots of alerts, but if you want to know about all failed logins to your tenant, just exclude this condition.

image

The second part of the filter is to show activities that equal ‘Failed log on’ as shown. You can customise this further if you want to make it more granular, but for now let’s track any failed login to the tenant.

image

So the final query should look like the above.

image

You should be not surprised to see the number of results you get as shown above. In my case, there are failed logins from the US, China, Russia, Italy, etc.

Now of course, I can drill into each item for more details but what I really want is a way for me to be alerted about these when they happen. Cloud App Security to the rescue again.

image

At the top of the results you should find a button as shown that says New policy from search, which you should select.

image

You should now see a page like shown above where you can define your policy. You’ll need to give it a name and description. You may want to increase the severity or change the category to suit. You can also select between single or repeated activity.

image

Now if you want an email alert then you’ll need to select the option Send alert as email and put in your email address as shown above. You may also want to change the Daily alert limit to suit your needs.

When you have completed the configuration, scroll to the bottom of the page and select the Create button.

image

You should now see that policy in the list in your tenant as shown above.

Cloud App Security is a really powerful tool that I believe is a must have for every Microsoft 365 tenant, because not only can you create your own custom queries but you can also convert those into alerts as I have shown.

Cloud App Discovery/Security

Microsoft has a range of security options available, delivered in a variety of ways from the cloud. I’m going to focus on three items that tend to get lumped together and with which I see much confusion. These services are:

1. Azure AD Cloud App Discovery

2. Office 365 Cloud App Security

3. Microsoft Cloud App Security

Here’s a summary of the differences between the products:

image

1. Azure AD Cloud App Discovery

This is the most basic of the three services and is only available when you purchase and license Azure AD P1, there is no stand alone version of just Azure AD Cloud App Discovery. This is a description of the product:

Azure Active Directory Premium P1 includes Azure Active Directory Cloud App Discovery at no additional cost. This feature is based on the Microsoft Cloud App Security Cloud Discovery capabilities that provide deeper visibility into cloud app usage in your organizations. Upgrade to Microsoft Cloud App Security to receive the full suite of Cloud App Security Broker (CASB) capabilities offered by Microsoft Cloud App Security.

Thus you receive Azure AD Cloud App Discovery when you purchase the following:

– Azure AD Premium P1 (stand alone)

– Azure AD premium p2 (stand alone)

– Microsoft 365 E3 (which includes Azure AD P1)

– Enterprise and Mobility Suite (EMS) E3 (which includes Azure AD P1)

When you visit the portal you will see:

image

Firstly, note that the banner reads Cloud App Security like so:

image

2. Office 365 Cloud App Security

This is available as a stand alone purchase for existing Office 365 / Microsoft 365 tenants.

clip_image001

You can also get Office 365 Cloud App Security as part of:

– Office 365 E5

You’ll see Office 365 Cloud App Security in the top left of the portal like so:

image

The biggest advantage I believe of Office 365 Cloud App Security over Azure AD Cloud App Discovery is the Activity policies like so:

image

These activities includes built in anomaly detection for things like Impossible travel like so:

image

You also get a number of default activity policies like Logon from a risky IP address:

image

as well as the ability to create your own unique activity policies and alerting.

3. Microsoft Cloud App Security

This again, is available as a stand alone add-on to any Office 365 / Microsoft 365 tenant, being a tad more expensive that office 365 Cloud App Security:

clip_image001[4]

It is also available when you purchase:

– Microsoft 365 E5

– Microsoft 365 E5 Security

– Enterprise and Mobility Suite (EMS) E5

As you can see from the table at the top of this article, Microsoft Cloud App Security includes everything (plus more) that is in Azure AD Cloud App Discover and Office 365 Cloud App Security. Thus, think of Microsoft Cloud App Security as Azure AD Cloud App Discovery + Office 365 Cloud App Security.

This is what you see when login to Dashboard:

image

You’ll see it look very different even though the top left says “Cloud App Security” again. You get far more options that with either of the other two including more options under Investigate like so:

image

Summary

Not everything is quite as simple as I have outlined here. Deeper detail about the licensing can be found here:

https://query.prod.cms.rt.microsoft.com/cms/api/am/binary/RE2NXYO

or on:

Microsoft Cloud App Security

In my opinion Azure AD P1 is a must have for all tenants to get features like conditional access and trusted IP’s. That will give you Azure AD Cloud App Discovery. However, I’d also recommend also adding Office 365 Cloud App Security as a minimum to get access to the Activity alerts. if you want even more power then add Microsoft Cloud App Security instead.

The final question I get is whether you require a license for all users in your tenant? For that I will leave you with the official word from Microsoft on that topic which is:

“Each user must be licensed for Microsoft Cloud App Security to use or benefit from it. For customers who license a subset of users, services enforced at the tenant level are not licensed for the other users. They are not entitled to use or benefit from the service, regardless of whether the service is technically accessible.”

Define an IP range in Cloud App Security

image

For me, Office 365 Cloud App Security is a must have add on for any Microsoft or Office 365 tenant as I have spoken about here:

A great security add on for Microsoft 365

As with all services, once you have enabled it you need to do some customisation to get the best from it. The first thing you should do is define your ‘corporate’ IP addresses. These typically refer to your on premises environment.

The first step in defining these is to access Office 365 Cloud App security, which you can do from the Microsoft 365 Security Center. Once at the home page, select the COG in the top right hand corner.

image

That should reveal a menu like you see above. From this menu select the option IP address ranges.

image

Then select the Category option in the middle of the page and the option for Corporate.

image

You will then see an IP address ranges that have been defined as ‘corporate’ already. To add more ranges simply select the + (plus) button in the upper right. Doing show will provide you a dialog box like shown above where you can now enter the appropriate details.

Why is defining your ‘corporate’ IP addresses important? It helps prevent false positives, especially when you have multiple locations. This is handy when you start setting up rules in Office 365 Cloud App Security, you can easily use the ‘corporate’ definition to designate your known environment. It means also that when you add new locations you don;t have to go and change all your rules, just add top the ‘corporate’ IP range list.