Determining the time Office 365 ATP takes to scan an attachment

Office 365 Advanced Threat Protection (ATP) has the ability to sandbox and test attachments prior to delivery to an Office 365 inbox. This is known as ATP Safe Attachments which you read about here:

Office 365 ATP Safe Attachments

Basically, it takes email attachments and opens them in a protected sandbox inside the Microsoft data center to see whether they do any malicious or unexpected. If it does, then actions can be taken to prevent that attachment from reaching the inbox. If not, the attachment is delivered as normal.

Now this sandbox testing does cause a slight delay in delivery of attachment. In my experience, I have never seen any attachment, no matter how large take longer than 2 minutes to deliver. However, there maybe the need to test this delivery time when troubleshooting.

Luckily, I looked around and found this great article from Kloud:

https://blog.kloud.com.au/2018/07/19/measure-o365-atp-safe-attachments-latency-using-powershell/

which contains some handy scripting to allow you to determine the time ATP takes to verify an attachment. So I thought I’d build on that.

To complete this process you firstly need to have a tenant that has Office 365 ATP assigned to it. You’ll also need to target a recipient that has an Office 365 ATP license assigned to them. You’ll basically send this recipient two emails, one with an attachment and one without, and then we’ll use a script to determine and report the time difference.

So step 1 is to send a standard email without an attachment to the recipient. I’ll do this here from my Yahoo account.

Once that has been successfully sent, I’ll immediately send another email that is basically the same but this time with an attachment. In this case, I’m send a Word document of 52KB in size.

I need to now wait to ensure both emails are FULLY delivered to the recipient.

If you have Safe Attachment Dynamic Delivery enabled where the body is received while the attachment is still being scanned you need to wait until this scanning process has FULLY completed.

That is, you need to wait until the whole message, including the attachment has been delivered to the Inbox as shown above.

Ensure that you are connected to Exchange Online with PowerShell already and then run my script, which you can find at:

https://github.com/directorcia/Office365/blob/master/o365-atp-timer.ps1

After a few moments you should see the results like that shown above, giving you the number of additional second it took to scan the attachment. In this case around 101 seconds.

There is no real guidance from Microsoft on how long ATP scanning should take so if you do run this script I’d really appreciate you completing this short survey:

ATP Timings

so we can get an idea of what people are seeing out there with ATP. That should also give us an ‘average’ figure we can use to understand ‘normal’ ATP performance.

The survey has one required field of the time in seconds you received but if you could also indicate the size of the attachment you tested that would also help understand whether the size of attachment play a role in any way.

Like I said, my experience has been that ATP never takes more than around 2 minutes to do attachment scanning but I’d love to get your feedback in the survey if you run this script. Thanks again to Kloud for their blog post around this and doing the hard scripting yards.