The case of the missing Azure Sentinel ingested data

image

Recently, I have seen my Azure Sentinel overview look like the above. I was puzzled why I had so many hours without any data being ingested? In short, it turned out that I had exceeded my storage tier capacity. Here’s where to look if you see something similar.

image

From the menu on the left of the Azure Sentinel workspace scroll to the bottom and select Settings as shown. Then from the pane that appears on the right select Workspace settings at the top as shown.

image

This will take you to the Azure Log Analytics workspace that underpins Sentinel. From the menu on the left here select Usage and estimated costs. Note on the right what is highlighted under Free pricing tier I was using:

(The log data ingestion includes the 500 MB/VM/day data allowances from Azure Security Center.)

That is the limit for my current tier. Any ingested data over that quota was not being ingested. Not ingested data, nothing recorded in the Sentinel overview report.

image

If you select the Daily cap button at the top of the page you’ll get more information appear from the right as shown.

image

The two important things to note are that the daily volume cap is 0.5 GB/day and that the limit is reset at 2am UTC (12pm Sydney time).

clip_image001

When I checked the Workspace Pricing tier details, shown above, there is indeed a daily cap of 512MB.

image

Then when I looked at the overview report in Sentinel I see that data did indeed start begin re-ingest at 12pm local time (2am UTC) as expected.

image

So the next question was, how is it going to cost me avoid this situation and ingest all my data? Looking at the Pay-as-you-go pricing tier I see the estimated cost per month would only be AU$4.79. Easy choice.  SELECT.

SNAGHTMLd5b1a0

The important thing to remember with this ingested data is that you always get the initial 512MB per day free. Anything above that you won’t get any captured data unless you upgrade your pricing tier. But then you’ll only pay for the amount above the 512MB per day, which in my case was only about 34MB per day on average.

image

A good way to keep track of this sort of data, before it becomes and issue as it did for me, is to use the Workspace usage Report workbook which you can access from the Sentinel console as shown above.

image

Here you’ll see everything you need to keep on top of this total data you are ingesting and where it is coming from.

The reason I’m so much data is that I’m pulling security events from local devices. Most Microsoft cloud services include free ingestion, which is the place you should start. However, I had added a number of demo devices to my tenant which pushed me over the free 512MB limit. Most people should be able to stay well below this quota by default, at least to start with. However, if you ever need to upgrade, like I have, it’s still cheap for it provides!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s