End to End email protection with Microsoft 365–Part 6

This is part of a series of articles about email security in Microsoft 365. Please check out previous articles here:

End to End email protection with Microsoft 365 – Part 1

End to End email protection with Microsoft 365 – Part 2

End to End email protection with Microsoft 365 – Part 3

End to End email protection with Microsoft 365 – Part 4

End to End email protection with Microsoft 365 – Part 5

These articles are based on a model I have previously created, which you can read about here:

CIAOPS Cyber protection model

designed to help better explain expansive security included with Microsoft 365.


Email reporting and auditing

It’s now time to look at all the logging that occurs during even the simply process of receiving and viewing an email. For starters there is:

Message tracing

and

Message trace in the modern Exchange admin center

Message trace in the Security & Compliance Center follows email messages as they travel through your Exchange Online organization. You can determine if a message was received, rejected, deferred, or delivered by the service. It also shows what actions were taken on the message before it reached its final status.

There is also reporting options like:

Mail flow insights in the Security & Compliance Center

and

Mail flow reports in the Reports dashboard in Security & Compliance Center

as well as:

Microsoft 365 Reports in the admin center – Email activity

If you want to specifically look at email security there is:

Email security reports in the Security & Compliance Center

as well as:

Defender for Office 365 reports in the Reports dashboard in the Security & Compliance Center

and

Reports for data loss prevention (DLP)

I have also spoken about the importance of the Unified Audit Logs (UAL) in Microsoft 365:

Enable activity auditing in Office 365

Unified Audit Logs in Microsoft 365

and you need to ensure that these have been enabled so that you can:

View mailbox auditing

Starting in January 2019, Microsoft is turning on mailbox audit logging by default for all organizations. This means that certain actions performed by mailbox owners, delegates, and admins are automatically logged, and the corresponding mailbox audit records will be available when you search for them in the mailbox audit log.

Here are some benefits of mailbox auditing on by default:

  • Auditing is automatically enabled when you create a new mailbox. You don’t need to manually enable it for new users.

  • You don’t need to manage the mailbox actions that are audited. A predefined set of mailbox actions are audited by default for each logon type (Admin, Delegate, and Owner).

  • When Microsoft releases a new mailbox action, the action might be automatically added to the list of mailbox actions that are audited by default (subject to the user having the appropriate license). This means you don’t need to monitor add new actions on mailboxes.

  • You have a consistent mailbox auditing policy across your organization (because you’re auditing the same actions for all mailboxes).

With this auditing enabled you can do things like:

Reporting mailbox logins

and

Search the Office 365 activity log for failed logins

as well as

Audit Office 365 user logins via PowerShell

Many of the reports that you find in the Microsoft 365 Admin area can be scheduled to be sent via email per:

Scheduling compliance reports

Apart from auditing and security you can also do more typical things like:

Viewing mailbox usage

Viewing Email apps usage

The availability of all this data is covered here:

Reporting and message trace data availability and latency

typically being 90 days.


User reporting and auditing

For information more specifically about user logins into the service and the Identity container, the best place to look is in Azure Active Directory (AD).

What are Azure Active Directory reports?

Find activity reports in the Azure portal

Azure Active Directory sign-in activity reports – preview

Audit activity reports in the Azure Active Directory portal

and if you want use PowerShell

Azure AD PowerShell cmdlets for reporting

Device reporting and auditing

There are lots of options when it comes to monitoring and reporting on devices. Apart from what is offered locally you also have:

Intune report

Create diagnostic settings to send platform logs and metrics to different destinations

Manage devices with endpoint security in Microsoft Intune

You can even get telemetry data and analytics reports from your desktop applications via:

Windows Desktop Application Program


Aggregated data reporting and monitoring

As you can see with all the options above, it is easy to get to information overload trying to keep up with all those signals. Luckily Microsoft provides a range of services to aggregate all this for you to make monitoring and report easier.

The first is Microsoft Cloud App Security services:

Cloud App Discovery/Security

Microsoft Cloud App Security overview

Microsoft Cloud App Security data security and privacy

There are plenty of reasons why you really should have Microsoft Cloud App Security in your environment:

A great security add on for Microsoft 365

Office 365 Cloud App Discovery

Next, is Microsoft Defender for Endpoint that will aggregate security and threat information for devices in your environment and make it available in a single console.

Overview of Microsoft Defender Security Center

Microsoft Defender Security Center portal overview

Microsoft Defender for Endpoint

Microsoft Defender for Endpoint evaluation lab

Finally for me, there is Azure Sentinel, which I see as really the ultimate hub for event reporting, monitoring and alrtign across the whole service.

Another great security add on for Microsoft 365

Introduction to Azure Sentinel

Azure Sentinel is a service that growing in features rapidly:

A couple of new additions to Azure Sentinel

Stay ahead of threats with new innovations from Azure Sentinel


Summary

Hopefully, all this gives you some insight into all the auditing and usage data that Microsoft 365 captures during any interaction within the service. One of the biggest benefits is also how this information is integrated between services, especially those that aggregate information lime Microsoft Cloud App Security and Azure Sentinel. This means you don’t have to crawl through individual log entries, you can use a dashboard and drill down from there. I also like the fact that all of these services and data are accessible using a scripting tool like PowerShell if you want to automate this further.

Remember, throughout this six part series I’ve just looked at what happens when a single email is delivered and view with Microsoft 365. If you expand that out to all the services and capabilities that Microsoft 365 provides you can hopefully get a better appreciate of the protection it provides in place for your data on many different levels.

The call to action for readers is to go away and implement all the security features that Microsoft 365 provides. This may of course vary by the license that you have. You should then consider what additional security offerings the Microsoft cloud stack can offer that makes sense for your business, then implement those. Remember, security is not a destination, it is journey.

Azure AD Sign-in error code look up

image

When you are looking at various entries in the Azure AD logs you will find, under the Basic Info tab, a Sign-in error code and directly below that a Failure reason field as shown above.

image

The above, shows you these fields in more detail.

You may not be aware but if you navigate to the web site:

https://login.microsoftonline.com/error

image

and plug in the Sign-in error code from the event, you should see information like that shown above. Most of it should match what the Failure reason field says. There can however, also be additional information in there that may help you when it comes to troubleshooting these events.


Office 365 Audit Retention Policy

I have spoken previously about the importance of ensuring that your unified audit logs are enabled in your Microsoft 365 tenant:

Enable activity auditing in Office 365

These logs are retained for 90 days by default for all plans. However, if you have Office 365 E5, Microsoft 365 E5 or Microsoft 365 E5 Compliance add-on license you can enable an audit retention policy for up to 1 year.

If you navigate to:

https://protection.office.com/unifiedauditlog

in your tenant you will see:

image

the button New audit retention policy at the bottom of the page as shown above.

image

Select that button will display the above dialog. Towards the bottom of this you will see that you can set up a retention policy of up to 1 year.

Of course you can enter the policy via the web interface but I prefer PowerShell. The command that you need to use is:

New-UnifiedAuditLogRetentionPolicy

you then use the recordtypes parameter to specify the audit logs of a specific record type that are retained by the policy. Currently, there are heaps of these:

  1. AeD
  2. AirInvestigation
  3. ApplicationAudit
  4. AzureActiveDirectory
  5. AzureActiveDirectoryAccountLogon
  6. AzureActiveDirectoryStsLogon
  7. CRM
  8. Campaign
  9. ComplianceDLPExchange
  10. ComplianceDLPSharePoint
  11. ComplianceDLPSharePointClassification
  12. ComplianceSupervisionExchange
  13. CustomerKeyServiceEncryption
  14. DLPEndpoint
  15. DataCenterSecurityCmdlet
  16. DataGovernance
  17. DataInsightsRestApiAudit
  18. Discovery
  19. ExchangeAdmin
  20. ExchangeAggregatedOperation
  21. ExchangeItem
  22. ExchangeItemAggregated
  23. ExchangeItemGroup
  24. HRSignal
  25. HygieneEvent
  26. InformationBarrierPolicyApplication
  27. InformationWorkerProtection
  28. Kaizala
  29. LabelExplorer
  30. MIPLabel
  31. MailSubmission
  32. MicrosoftFlow
  33. MicrosoftForms
  34. MicrosoftStream
  35. MicrosoftTeams
  36. MicrosoftTeamsAdmin
  37. MicrosoftTeamsAnalytics
  38. MicrosoftTeamsDevice
  39. MicrosoftTeamsShifts
  40. MipAutoLabelExchangeItem
  41. MipAutoLabelSharePointItem
  42. MipAutoLabelSharePointPolicyLocation
  43. OfficeNative
  44. OneDrive
  45. PowerAppsApp
  46. PowerAppsPlan
  47. PowerBIAudit
  48. Project
  49. Quarantine
  50. SecurityComplianceAlerts
  51. SecurityComplianceCenterEOPCmdlet
  52. SecurityComplianceInsights
  53. SharePoint
  54. SharePointCommentOperation
  55. SharePointContentTypeOperation
  56. SharePointFieldOperation
  57. SharePointFileOperation
  58. SharePointListItemOperation
  59. SharePointListOperation
  60. SharePointSharingOperation
  61. SkypeForBusinessCmdlets
  62. SkypeForBusinessPSTNUsage
  63. SkypeForBusinessUsersBlocked
  64. Sway
  65. SyntheticProbe
  66. TeamsHealthcare
  67. ThreatFinder
  68. ThreatIntelligence
  69. ThreatIntelligenceAtpContent
  70. ThreatIntelligenceUrl
  71. WorkplaceAnalytics
  72. Yammer

In my case I ran:

New-UnifiedAuditLogRetentionPolicy -Name “Log Retention Policy” -Description “One year retention policy for all activities” -RecordTypes AeD,AirInvestigation,ApplicationAudit,AzureActiveDirectory,AzureActiveDirectoryAccountLogon,AzureActiveDirectoryStsLogon,CRM,Campaign,ComplianceDLPExchange,ComplianceDLPSharePoint,ComplianceDLPSharePointClassification,ComplianceSupervisionExchange,CustomerKeyServiceEncryption,DLPEndpoint,DataCenterSecurityCmdlet,DataGovernance,DataInsightsRestApiAudit,Discovery,ExchangeAdmin,ExchangeAggregatedOperation,ExchangeItem,ExchangeItemAggregated,ExchangeItemGroup,HRSignal,HygieneEvent,InformationBarrierPolicyApplication,InformationWorkerProtection,Kaizala,LabelExplorer,MIPLabel,MailSubmission,MicrosoftFlow,MicrosoftForms,MicrosoftStream,MicrosoftTeams,MicrosoftTeamsAdmin,MicrosoftTeamsAnalytics,MicrosoftTeamsDevice,MicrosoftTeamsShifts,MipAutoLabelExchangeItem,MipAutoLabelSharePointItem,MipAutoLabelSharePointPolicyLocation,OfficeNative,OneDrive,PowerAppsApp,PowerAppsPlan,PowerBIAudit,Project,Quarantine,SecurityComplianceAlerts,SecurityComplianceCenterEOPCmdlet,SecurityComplianceInsights,SharePoint,SharePointCommentOperation,SharePointContentTypeOperation,SharePointFieldOperation,SharePointFileOperation,SharePointListItemOperation,SharePointListOperation,SharePointSharingOperation,SkypeForBusinessCmdlets,SkypeForBusinessPSTNUsage,SkypeForBusinessUsersBlocked,Sway,SyntheticProbe,TeamsHealthcare,ThreatFinder,ThreatIntelligence,ThreatIntelligenceAtpContent,ThreatIntelligenceUrl,WorkplaceAnalytics,Yammer -RetentionDuration TwelveMonths -Priority 100

to set them all for my E5 environment, and thus retain all this logging information for at least 12 months!

image

You can read more about all this in the Microsoft documentation here:

Manage audit log retention policies

Remember however, for this to work:

“To retain an audit log for longer than 90 days, the user who generated the audit log must be assigned an Office 365 or Microsoft 365 E5 license or have a Microsoft 365 E5 Compliance add-on license.”

***** 9 April 2020 Update

It appears Microsoft has now changed the parameters you can specify to:

ExchangeAdmin, ExchangeItem, ExchangeItemGroup, SharePoint, SyntheticProbe, SharePointFileOperation,
OneDrive, AzureActiveDirectory, AzureActiveDirectoryAccountLogon, DataCenterSecurityCmdlet,
ComplianceDLPSharePoint, Sway, ComplianceDLPExchange, SharePointSharingOperation,
AzureActiveDirectoryStsLogon, SkypeForBusinessPSTNUsage, SkypeForBusinessUsersBlocked,      SecurityComplianceCenterEOPCmdlet, ExchangeAggregatedOperation, PowerBIAudit, CRM, Yammer,      SkypeForBusinessCmdlets, Discovery, MicrosoftTeams, ThreatIntelligence, MailSubmission, MicrosoftFlow,  AeD, MicrosoftStream, ComplianceDLPSharePointClassification, ThreatFinder, Project,  SharePointListOperation, SharePointCommentOperation, DataGovernance, Kaizala, SecurityComplianceAlerts, ThreatIntelligenceUrl, SecurityComplianceInsights, MIPLabel, WorkplaceAnalytics, PowerAppsApp,  PowerAppsPlan, ThreatIntelligenceAtpContent, LabelContentExplorer, TeamsHealthcare, ExchangeItemAggregated, HygieneEvent, DataInsightsRestApiAudit, InformationBarrierPolicyApplication,   SharePointListItemOperation, SharePointContentTypeOperation, SharePointFieldOperation,  MicrosoftTeamsAdmin, HRSignal, MicrosoftTeamsDevice, MicrosoftTeamsAnalytics, InformationWorkerProtection,  Campaign, DLPEndpoint, AirInvestigation, Quarantine, MicrosoftForms, ApplicationAudit,  ComplianceSupervisionExchange, CustomerKeyServiceEncryption, OfficeNative, MipAutoLabelSharePointItem,     MipAutoLabelSharePointPolicyLocation, MicrosoftTeamsShifts, MipAutoLabelExchangeItem, CortanaBriefing,
Search, WDATPAlerts, MDATPAudit