My Tech Books – 2020

Tech is as much a lifestyle choice these days as it is a career. The geeks and nerds have risen to rule the world. Don’t believe me? Ask Bill Gates! Sometimes it is good to step back and take a wide look at how technology has changed the world we live in – for better and worse. My selection below I have found to be enjoyable and thought provoking in many different ways and I recommend them to everyone who is interested in tech.

There hasn’t been an change to this since last year. Good tech books are hard to come by it seems!

You can follow all the books, tech, business, non-fiction I read and want to read over at Goodreads where I have an account. You can also view my activity via:

https://www.goodreads.com/director_cia


1. Daemon – Daniel Suarez [Fiction]

A glimpse into the future of where drones and augmented reality may take us. That may not necessarily be a good place either.

2. Freedom TM – Daniel Suarez [Fiction]

A follow up to Daemon. What happens when technology dominates the world? Who benefits?

3. Ready Player One – Ernest Cline [Fiction]

Much like the Matrix. What is life like if you live inside the machine? You can be just about anyone you choose. I also love this book for all the retro technology that was part of my life. TRS-80 anyone? This book has become so popular that there is now a movie. Believe me, the book is better.

4. Future Crimes: Inside the Digital Underground and the Battle for our Connected World – Marc Goodman [Non-fiction]

Technology will ultimately doom us all I believe because we are building our world on stuff that unfortunately places a low regard for security and privacy. This book will show you why that is a road to ruination.

5. Countdown to Zero Day: Stuxnet and the Launch of the World’s First Digital Weapon – Kim Zetter [Non-Fiction]

If you don’t believe cyber warfare is real then read this book to understand how software is now a weapon as potentially devastating as any nuclear device.

6. Beyond Fear: Thinking Sensibly about Security in an Uncertain World – Bruce Schneier [Non-Fiction]

Security is important but it is important in context. We need to be rational when we consider our security not emotional. A great level headed approach to how we need to be secure.

7. American Kingpin: The Epic Hunt or the Criminal Mastermind Behind the Silk Road – Nick Bilton [Non-Fiction]

An amazingly detailed book on the rise and fall of Ross Ulbricht, the creator of the Silk Road web site. In here are asked to think about whether technology plays something more than a neutral role in today’s world.

8. The Cuckoos Egg – Clifford Stoll [Non-Fiction]

Before the Internet was in the public sphere it existed in the world of academia. This is the story of how one man’s search for the source of an accounting error uncovered something are more sinister.

9. Takedown – John Markoff and Tsutomu Shimomura [Non-Fiction]

The pursuit and eventual capture of notorious hacker Kevin Mitnick makes for great reading.

10. Hackers: Heroes of the Computer Revolution – Steven Levy [Non-Fiction]

Ah, the good ole days when it was more about proving how smart you were than trying to actually cause harm. If you think hacking is something new, then you’re in for a surprise with this book

Moving to the Cloud–Part 3

This is part 3 of a multi part examination of moving to the Microsoft cloud. If you missed the first episode, you’ll find it here:

Moving to the Cloud  – Part 1

which covered off setting up a site to site VPN to Azure and

Moving to the Cloud – Part 2

which looked at creating traditional ‘dive mapped’ storage as PaaS.

It is now time to consider identity. We need to know where a user’s identity will live in this new environment because there are a few options. Traditionally, a user’s identity has lived on premises in a local domain controller (DC) inside an Active Directory (AD). With the advent of the cloud we now have Azure Active Directory (AAD) as an option as well. It is important here to remember that Azure Active Directory (AAD) is NOT identical to on premises Active Directory (AD) per:

image

What this means is that native Azure AD (AAD) can’t do some things that on premises Active Directory (AD) can do. Much of that is legacy services like Group Policy and machine joins, etc. You’ll see that Windows 10 machines can be joined to Azure AD (AAD) directly but legacy systems, like Windows 7, 8 and Windows Servers can’t be directly joined to AAD. That’s right. As we stand today, even the latest Windows Server cannot be directly joined to AAD like it can be joined to an AD on premises.

Thus, if you have legacy services and devices as well as Windows Servers you want to remain as part of your environment, you are going to need to select an identity model here that supports traditional domain joins. I will also point out that, as of today (changing in the future), if you want to implement Windows Virtual Desktop (WVD), you will also need a traditional AD to join those machines to. However, if you have no devices that require legacy services, for example if your environment is totally Windows 10 pro based with no servers (on prem or in Azure IaaS), then all you will need is Azure AD.

Thus, not every one can jump directly to AAD immediately. Most will have to transition through some form of hybrid arrangement that supports both AAD and AD in the interim. However, most transitions are ultimately aimed at eliminating on premises infrastructure to limit costs such as patching and updating things like physical servers. This will be what we are aiming for in this scenario.

In a migration from a traditional on premises environment with a domain controller (DC) and AD we now have a number of options when it comes to identity in the cloud.

1. You can maintain the on premises domain controller and AD, while using Azure AD Connect to synchronise (i.e. copy) the user’s identity to the AAD. It is important to note here that the identity in Azure is a COPY and the primary identity remains on premises in the local AD. This is still the case if you implement things like password write back that are part of Azure AD P1 and Microsoft 365 Business. Having the user’s primary identity still on premises means this is where you need to go to make changes and updates.

2. You can swing the domain controller from on premises to Azure IaaS. This basically means setting up a new VM in the Azure VNET that has been created already, joining it to the existing on premises domain across the VPN, then using DCPromo to make it a domain controller. To make it the ‘primary’ domain controller, you swing across the domain infrastructure roles via the following in PowerShell:

Move-ADDirectoryServerOperationMasterRole -Identity “Target-DC” -OperationMasterRole SchemaMaster,RIDMaster,InfrastructureMaster,DomainNamingMaster,PDCEmulator

and then DCPromo the original on premises domain controller out and then remove it altogether. This way you now have your Domain Controller and AD on the VM in Azure IaaS working with machines in the Azure VNET and on premises thanks to the site to site VPN established earlier (told you it would be handy!). In essence, this is like picking up the domain controller hardware and moving it to a new location. Nothing else changes. The workstations remain on the same domain, group policy is unaffected, etc, etc. The downside is that you still need to continue to patch and update the new domain controller VM in Azure but the maintenance and flexibility is superior now it is in Azure IaaS.

3. You replace the on premises domain with Azure AD Domain Services. Think of this like a cloud domain controller as a service. It is a Domain Controller as PaaS. This means that when you use Azure AD Domain Services Microsoft will spin up two load balanced domain controller VMs and connect this directly to AAD so the users there now appear in the PaaS domain controllers. Using Azure AD Domain Services removes the burden of you having to patch, update, scale, etc domain controllers for your environment. It also gives you a traditional AD environment you can now connect things like servers to. However, there are some trade offs. When you use Azure AD Domain Services you must start a new domain. This means you can’t swing an existing domain across onto it, like you can in step 2 above. This means detaching and reattaching all your legacy devices, like servers, from the original to new domain. You also get limited functionality with traditional AD services like Group Policy. You should see Azure AD Domain Services as a transitionary step, not an end point.

With all that in mind, you need to make a decision on what works best for your environment, now and in the future. Considering that most environments I see want to eliminate the on premises domain controller hardware as soon as possible and not replicate this going forward. That desire therefore means a migration to PaaS using Azure AD Domain Services.

The first step in this process then is going to be to ensure that all your users are in Azure AD. The assumption here is that you have already set up your Microsoft 365 environment and the users are configured in Azure AD. If you retaining an on premises domain controller you’ll need to have set up Azure AD Connect to copy the user identities to Azure AD. Azure AD is where Azure AD Domain Services will draw it’s identities when it is installed, so the users need to be there first. Once the users appear in Azure AD, next step will be to set up Azure AD Domain Services. You can kind of think of a traditional on premises domain controller as somewhat being equivalent to Azure AD combined with Azure AD Domain Services.

Setting up Azure AD Domain Services is done via the Azure portal.

image

Login as a global administrator and locate Azure AD Domain Services and select that.

image

You’ll most likely find that no services are as yet configured. Select the Add option from the menu across the top as shown above.

image

You then need to complete the details. Here we face an interesting question, what should we call this new ‘traditional’ managed domain we are about to create with Azure AD Domain Services? Should it be the same as what is being used in Azure AD already?

image

image

How you configure this is totally up to you. There is guidance, as shown above, which can be found at:

Active Directory: Best Practices for Internal Domain and Network Names

In this case I have decided to go for a sub-domain, as recommended, and prefix the new Azure AD Domain Services with the letter ‘ds’ i.e. ds.domain.com.

image

With all the options completed, select Next – Networking to continue.

image

Unfortunately, you can’t configure Azure AD Domain Services on the same subnet that has service endpoints as you can see above. You’ll see this if you configured your Azure storage to use private endpoints as we have, which has been previously recommended.

If so, then you can select the Manage link below this box and simply add a new subnet to your Azure VNET and then use that to connect Azure AD Domain Services to.

image

Before you continue to Administration, ensure that you are adding Azure AD Domain Services to your existing Azure VNET as the default is to create a new VNET, which is NOT what you want here. You want to connect it to an existing VNET you have established previously.

When you have selected your existing Azure VNET and a suitable subnet, select the Next – Administration button to continue.

image

Here you’ll need to decide which users will be administrators for the domain.  So from the documentation:

What can an AAD DC Admin do?

Only Microsoft has domain admin and enterprise rights on the managed domain. AAD DC Admins can do the following:

  • Admins can use remote desktop to connect remotely to domain-joined machines

  • Admins can join computers to the domain

  • Admins are in the administration group on each domain-joined machine

Considerations for the AAD DC Administrators group

  • Pick group members for the AAD DC Administrators group that have these needs:

    • Users that need special administrative permissions and are joined to the domain

    • Users that need to join computers to the domain
  • Do not change the name of the AAD DC Administrators group. This will cause all AAD DC Admins to lose their privileges.

The default will be your global administrators and members of a special group called AAD DC Administrators, that will be created. So, you can simple add any Azure AD user to this group and they will have admin privileges in the  Azure AD Domain Services environment going forward.

You can of course configure these permissions any way you wish but generally the defaults are fine so select the Next – Synchronization button to continue.

image

The final question is whether you wish to have all or a subset of your Azure AD users synchronised into the Azure AD Domain Service environment. In most cases, you’ll want all users, so ensure that option is select and press the Review + create button to continue.

image
image

You should now see all your settings and importantly, note the box at the bottom about consenting to store NTLM and Kerberos authentication in Azure AD. This is because these older protocols have potential security concerns and having them stored in a place other than a  domain controller is something you need to be aware of. Generally, there won’t be any issues, but make sure you are aware of what that last box means for your security posture.

Press the Create button when complete.

image

You’ll then receive the above warning about what configurations options can’t be changed after the fact. Once you have reviewed this and you wish to proceed, select the OK button.

Your deployment into Azure will then commence. This process should generally take around 1 hour (60 minutes).

image

You should see the above message when complete and if you select Go to resource you’ll see:

image

You’ll note that it still says Deploying here, so you’ll need to wait a little longer until that process is complete.

image

In about another 15 minutes you should see that the domain is fully deployed as shown above. Here you will note that two domain controllers have automatically been allocated. In this case they are 10.0.1.5 and 10.0.1.6 on the subnet into which Azure AD Domain Services was deployed. You can select from a number menu options on the left but the service is pretty basic. Most times you’ll only need to look at the Activity log here from now on.

Can you actually manage the domain controllers like you can on premises? Yes, somewhat. To do that you’ll need to download and install the:

Remote Server Administration Tools for Windows 10

on a Windows 10 workstation that can access these domain controllers.

image

You can then use that to view your domain in the ‘traditional way’ as shown above.

Thus, with Azure AD Domain Services deployed, you have a ‘traditional’ domain but without infrastructure and with your Azure AD users in there as well.

The summary of the options around identity here are thus:

1. Primary = local AD, Secondary = none (which can be linked to Azure via a VPN)

2. Primary = Azure AD, Secondary = none (no on premises infrastructure like servers to worry about)

3. Primary = local AD, Secondary = Azure AD (thanks to Azure AD Connect, but need a VPN again to connect to Azure IaaS)

4. Primary = Azure AD, Secondary = Azure AD Domain Services (which can be linked backed to on premises via a VPN)

In this case, we’ll be going with Option 4. You can see however that a VPN is going to be required for options 1, 3 and 4. That’s why one of the first steps in this series was to set one up.

With all that now configured, let’s now look at the costs involved. The costs here will vary on what identity solution you select. If you stay with an on premises domain controller only, you will need to have site to site VPN to resources in Azure. The costing for this has been covered previously:

Moving to the Cloud  – Part 1

and equates to around AU$36 per month with less than 5GB of traffic inbound to Azure. Azure AD Connect software you use to synchronise user identities to Azure AD is free.

If you move the domain controller to a virtual machine in Azure, there will be the cost of that virtual machine (compute + disk storage). The cost will therefore vary greatly on what VM type you select. I’ll be covering more about VM options in this migration in an upcoming article, but for now let’s keep it simple and say we use a A2v2 Standard VM (4GB RAM, 20GB HDD) for a single role as just a domain controller. The cost for that is around AU$76 per month. If you also still have on premises infrastructure, like Windows Servers, that need access to the domain, then you’ll also need a site to site VPN to communicate with the domain controller VM in Azure IaaS. Thus, to move the domain controller to Azure IaaS and still allow access to on premises infrastructure the cost would be around AU$112 (Azure VM + VPN). Of course, if you can migrate all your on premises server infrastructure to Azure IaaS, you probably wouldn’t need the VPN but there would then be the costs of the additional infrastructure in Azure. Balanced against this cost in Azure IaaS is the saving in local hardware, power, etc.

Again, let’s keep it simple for now and say we want to maintain on premise infrastructure but have a dedicate domain controller in the Azure IaaS so the one on premises can be de-commissioned. That means the costs would be AU$112 per month for a domain controller in Azure IaaS and a VPN back to on premises.

Finally, the last identity option is if we wanted to use the Azure PaaS service, Azure AD Domain Services, which means no infrastructure at all but also means we need to start with a new ‘clean’ domain separate from the existing on premises one. The costs of this Azure PaaS service can be found at:

Azure Active Directory Domain Services pricing

which reveals:

image

For smaller directories (<25,000 objects) the cost is going to be AU$150 per month flat. Remember, here when equating costs, there are no VMs to backup or operating systems to patch because it is PaaS. This is a domain controller as a service and Microsoft will take care of all the infrastructure “stuff” for you as part of that service. Of course, if you need on premises infrastructure to access Azure AD Domain Services, you’ll again need a site to site VPN to get there. If all your infrastructure is cloud based, then no site to site VPN is required. However, in this scenario, we still want access to on premises infrastructure so the costs would be AU$186 per month (Azure AD Domain Services + VPN).

In summary then, the configuration options/costs will be:

Option 1. Retain on premises AD = AU$36

Option 2. Move domain controller to Azure IaaS = AU$112 (estimated typical cost)

Option 3. Migrate domain controller to Azure PaaS = AU$186 per month

Going forward we’ll be selecting Option 3, because we are aiming to minimise the amount of infrastructure to be maintained and we want to move to PaaS as soon as possible. That means the total cost of the migration so far is:

1. Site to Site VPN = AU$36

2. Storage = AU$107

3. Identity (PaaS) = AU$150

Total maximum infrastructure cost to date = AU$293 per month

This means we have:

1. Eliminated the old on premises domain controller (hardware, patching, backup, power, etc costs)

2. Can connect to on premises infrastructure to Azure AD (via Azure AD Domain Services and the VPN)

3. Have mapped tiered storage locations for things like archiving, profiles, etc that are PaaS

4. We can now build out a Windows Virtual Desktop environment

The next item that we’ll focus on is setting up a Windows Virtual Desktop environment as we now have all the components in place to achieve that.

Need to Know podcast–Episode 225

FAQ podcasts are shorter and more focused on a particular topic. In this episode I’ll talk about the recommended process for file migrations to Microsoft 365 collaboration.

Take a listen and let us know what you think – feedback@needtoknow.cloud

You can listen directly to this episode at:

https://ciaops.podbean.com/e/episode-225-process-for-file-migrations/

Subscribe via iTunes at:

https://itunes.apple.com/au/podcast/ciaops-need-to-know-podcasts/id406891445?mt=2

The podcast is also available on Stitcher at:

http://www.stitcher.com/podcast/ciaops/need-to-know-podcast?refid=stpr

Don’t forget to give the show a rating as well as send us any feedback or suggestions you may have for the show.

Resources

Process for file migrations to Microsoft 365

My Business Books – 2020

Not a lot has changed on this since last year which you can check out at:

My Business Books – 2019

I spent more time last year reading biographies of people like Douglas MacArthur and Winston Churchill that I did with business books. Worthwhile business books are become harder and harder to find these days in my opinion. All have something to offer but few have more than one concept or some really novel ground breaking idea in my opinion. I still have lots of business books to get through but I can say that I’m busting to read any on that list at this stage.

You can follow all the books I read and want to read over at Goodreads where I have an account. You can also view my activity via:

https://www.goodreads.com/director_cia

or just follow me on Facebook:

www.facebook.com/ciaops

Here’s my current top business books in order:

1. The Art of War – Sun Tzu

The all time classic on strategy. As relevant today as it ever was. A very short read but very deep.

2. The Millionaire Fastlane – M.J. DeMarco

I love the brutal honesty of this book. It doesn’t mince words about what it takes to shift from a pay check to actually living the life you want.

3. The Tipping Point – Malcolm Gladwell

The world is all about not what you know but who you know. This book explains exactly how this works and how to use it to your advantage.

4. The Four Hour Work Week – Tim Ferriss

Many people believe this book is about shirking responsibility. It is in fact a blueprint for how to free up your time to do things you want and enjoy. It will challenge the way you look at your career.

5. Secrets of the Millionaire Mind: Mastering the Inner Game of Wealth – T. Harv Eker

The successful are defined by a different mindset. This mindset can be learned. It can be trained. This is a great book to show you how to do just that.

6. Talent is over rated: What Really Separates World-Class Performers from Everyone Else – Geoff Colvin

Demonstrates that the best comes from implementing a system. Having a system allows you to focus on the right thing and do that work that is required. If you want to take yourself to an elite level, beyond just good, then read this book.

7. Book Yourself Solid: The Fastest, Easiest, and Most Reliable System for Getting More Clients Than You Can Handle Even If You Hate Marketing and Selling – Michael Port, Tim Sanders

You can’t survive in business without a steady flow of customers. Selling to people is the wrong approach, you instead need to attract them to your business. This book helps you achieve exactly that.

8. Profit First: A Simple System To Transform Any Business From A Cash-Eating Monster To A Money-Making Machine – Mike Michalowicz

Business is about making a profit. This then gives you the freedom to do what you want with that profit. This book helps you focus on profit and setting up systems to make the most of the profit you generate.

9. Barking Up the Wrong Tree – Eric Barker

Conventional wisdom does not always apply and in some case can actually be detrimental. Challenging what is taken for granted should be in the play book of everyone who wants to achieve at the highest level. Important lessons can be learned in the strangest places and form the strangest people. Have an open mind and you might be surprised at what you have believed to be bad in fact turns out to get just what you need.

10. Unbeatable Mind: Forge Resiliency and Mental Toughness to Succeed at an Elite Level – Mark Divine

Another mindset book. Business is not always going to be easy or take the intended route. This is when you need to have the determination to see your plans through to success. This book shows you how to develop the mental toughness to make this happen.

11. The E-Myth – Michael Gerber

The classic on ‘procedurising’ your business and creating a structure that doesn’t need you to survive. The simple secrets inside this book can transform any business from hardship to joy.

12. Tools of Titans – Tim Ferriss

There are few books that take the learnings for so many exceptional people and puts them at your fingertips. This is one such book that packs a lot of business and life learnings between the covers.

13. Predictably irrational: The Hidden Forces that Shape our Decisions – Dan Ariley

Although we like to think logic and rationality rule our world emotion is by far the more powerful influence. Understand this in the context of business and you are well on your way to understanding why people make the decisions they do and how to best profit from them.

14. Extreme Ownership – Jocko Willink and Lief Babin

Moving beyond blame is tough. This book illustrates the ownership of the problem and the environment is a key to success in the military or in business. It is a path few will elect to take voluntarily, however more may do so after reading this.

15. Peak Performance: Elevate your game, avoid burnout and thrive with the science of success – Brad Stulberg

Success is largely about developing a winning system. This book show you how to approach that pragmatically. If you want to see results use this book to help you build the system.

16. Blink: The Power of Thinking Without Thinking – Malcolm Gladwell

The older you get the more experience you get. This experience is aggregated in your ‘gut feel’. Trusting your ‘gut’ may not appear rational but this book will help you understand why it is in fact your best option in many cases.

17. The Now Habit: A Strategic Program for Overcoming Procrastination and Enjoying Guilt-Free Play – Neil A. Fiore

Plenty of great productivity learnings in here that help you take action. It shows you how to focus on the right stuff in the right priority. Even if you are not a major procrastinator there is plenty in this book that you can take away.

18. The One Thing – Gary Keller

Multi-tasking is a myth. Focus is the key to success to bringing all your resources to bear in unison makes a hell of a lot of difference. Most people can’t do it, so those that can stand a much greater chance of success.

19 Deep Work – Cal Newport

Distractions are wasted energy and time that you’ll never get back. You’d be amazed at how distracting the modern world is. If you can minimise these distractions you can focus more and be far more productive.

I’ve decided to spend more time with Audible books this year to help me get through more content. I can listen at increased speeds to get through more content and I can also listen in more locations with Audible books. I still enjoy reading ‘old style’ on my Kindle but there seem to be so many things that get in the way. Even if I am tired and worn down, listening to Audible is easy to do and generally more relaxing. I am hoping to churn through a lot more books this year in that way.

Let me know what you think. Do these work for you? What’s your top business reads? I’d love to hear.

CIAOPS Need to Know Microsoft 365 Webinar–January

laptop-eyes-technology-computer

We are back for a new decade! Yes, it is 2020 and Microsoft 365 webinars continue. This month I’m going to take a look of the some less common services of Microsoft 365 including Forms, To-Do, Whiteboard, Kaizala and more. I’ll have the  the latest Microsoft Cloud updates plus open Q and A as well. Start your year and decade off with a BANG!

You can register for the regular monthly webinar here:

January Webinar Registrations

The details are:

CIAOPS Need to Know Webinar – January 2020
Thursday 30th of January 2020
10.30am – 11.30am Sydney Time

All sessions are recorded and posted to the CIAOPS Academy.

The CIAOPS Need to Know Webinars are free to attend but if you want to receive the recording of the session you need to sign up as a CIAOPS patron which you can do here:

http://www.ciaopspatron.com

or purchase them individually at:

http://www.ciaopsacademy.com/

Also feel free at any stage to email me directly via director@ciaops.com with your webinar topic suggestions.

I’d also appreciate you sharing information about this webinar with anyone you feel may benefit from the session and I look forward to seeing you there.

Moving to the Cloud–Part 2

This is part of a multi part examination of the options of moving to the Microsoft cloud. If you missed the first episode, you’ll find it here:

Moving to the Cloud  – Part 1

which covered off setting up a site to site VPN to Azure.

The next piece of the puzzle that we’ll add here is storage.

Storage in the Microsoft cloud comes in many forms, SharePoint, Teams, OneDrive for Business and Azure. We’ll get to stuff in Microsoft 365 like SharePoint, Teams and OneDrive later, but to start off with we want to take advantage of the site to site VPN that was set up in Part 1.

In Azure there are three different access tiers of storage; hot, cool and archive. They all vary by access speed and cost. The slower the access, the cheaper it is. Hot is the fastest access, followed by cool, then archive. You can read more about this here:

Azure Blob storage: hot, cool, and archive access tiers

The other variable here with Azure storage is the performance tier; standard or premium. You can read more here:

Introduction to Azure storage

Basically, standard performance tier uses HDD while Premium uses SSD. Apart from performance, the major difference is how the storage cost is actually calculated. With the standard tier, you are only billed for the space you consume BUT you are also billed for access (read, write, delete) operations. With premium, you are billed for the total capacity of the storage you allocate immediately BUT, you are not billed for any access operations.

So the key metrics you need to keep in mind when you are designing a storage solution in Azure is firstly the access tier (hot, cool or archive) the performance tier (standard or premium) and the capacity you desire for each. You may find some combinations are unavailable, so check out the document linked above for more details on what is available with all these options.

The easiest approach to Azure storage is to create an Azure SMB Share and map these directly on a workstation which I have previously detailed here:

Creating an Azure SMB Share

as well as an overview on pricing:

Clarification on Azure SMB file share transactions

Azure SMB files currently only supports hot and cool tiers. You can use archive storage but only via blob access, not SMB files. So what good are all of these you may ask? Well, if you read my article:

Data discovery done right

You’ll find that I recommend dividing up your data into items to be deleted, archived and to be migrated.

So we need to ask ourselves the question, what data makes sense where?

Let’s start with Azure archive storage. What makes sense in here, given that Azure archive storage is aimed at replacement of traditional long term storage (think tape drives)? Into this, you want to put data that you aren’t going to access very often, and that doesn’t make sense going into Teams, SharePoint and OneDrive. What sort of data doesn’t make sense going into SharePoint? Data that can’t be indexed such as large image files without text, Outlook PST backups, custom file types SharePoint indexing doesn’t support (think some types of CAD files and other third party file types). In my case, Azure archive storage is a great repository for those PST backups I’ve accumulated over the years.

Here is the guidance from Microsoft:

  • Hot – Optimized for storing data that is accessed frequently.

  • Cool – Optimized for storing data that is infrequently accessed and stored for at least 30 days.

  • Archive – Optimized for storing data that is rarely accessed and stored for at least 180 days with flexible latency requirements (on the order of hours).

We now repeat this again but with the cool tier storage, remember that this tier now directly supports Azure SMB files. So, what makes sense here? There is obviously no hard and fast rule but again, what doesn’t make sense going into SharePoint? Stuff that can’t be indexed, is typically large, is not accessed that often but more often than archive storage AND you also want to be accessible via a mapped drive letter. In my case, that data that springs to mind are my desktop utility apps (like robocopy), ISO images (of old versions of SharePoint server I keep in case I need to do a migration) and copies of my podcast recordings in MP3 format.

We repeat this again for the hot tier which is fastest and most expensive storage option. Initially here I’m going to place the user profile data when I get around to configuring Windows Virtual Desktop (WVD) in this environment. That needs to be quick, however most other current data files I have will go into Microsoft 365. Being the most expensive tier of storage, I want to keep this as small as possible and only REALLY put data on here that makes sense.

You don’t have to use all three tiers as I do. You can always add more storage later if you need to, but I’d recommend you work out what capacity you want for each tier and then implement it. For me, I’m going for 100GB Archive, 100GB cool and 50GB hot as a starting point. Your capacities will obviously vary depending on how much data you plan to put in each location. That why you need to have some idea idea where all your data is going to go BEFORE you set all this stuff up. Some will go to Azure, some will go to Microsoft 365, some will deleted and so on.

As for performance tiers, I’m going to stick with standard across all storage accounts for now to keep costs down and only pay for the capacity I actually use.

Let’s now look at some costs by using the Azure pricing calculator:

image

I’ll firstly work out the price for each based on 1TB total storage for comparisons between the tiers and to SharePoint and OneDrive for Business.

All the storage calculations are in AU$, out of the Australian East data center, on the standard performance tier and locally redundant unless otherwise stated.

You can see that 1TB or archive storage is only AU$2.05, but it ain’t that simple.

image

There are other operations, as you can see above that need to be taken into account. I have adjusted these to what I believe makes sense for this example but as you can see, variations here can significantly alter the price (especially the read operations).

The estimated total for 1TB of archive storage on the standard performance tier = AU$27.05 per month.

Now, as a comparison, if I change the performance tier to Premium I get:

image

The price of the storage goes way up, while the price of operations goes way down. So, if you want to minimise costs and you have lots of operations on your storage, standard tier is your best option.

The estimated total for 1TB of archive storage on the premium performance tier = AU$224.22 per month.

Basically 10 x the cost above the standard tier.

In my case, I don’t need 1TB of storage, I only want 100GB of storage.

image

When I now do the estimation of 100GB of archive storage, the cost of just the storage falls by 10x (as expected) to AU$0.20, Don’t forget however about the storage operations which remain the same. So, my storage cost went down but my operation costs remained the same. Thus,

The estimated total for my 100GB of archive storage on the standard performance tier = AU$25.95 per month.

While premium is:

image

The estimated total for my 100GB of archive storage on the premium performance tier = AU$22.78 per month.

As outlined before, as a general rule of thumb with archive storage, premium performance tier is better value for low storage capacity and also low data operations. Once the capacity increases with premium performance, the price ramps ups.

So why would I recommend staying with the standard performance tier? Although, I ‘estimate’ that my archive will be small, I want the flexibility to grow the capacity if I need it. Remember, that we don’t set a storage capacity quota for block storage, it can just grow as needed and the bigger the storage capacity the more it will cost me if I go premium. Given that storage capacity here is more important than working with the data, I want the cheapest storage costs I can get as the data capacity increases. Thus, I’ll stick with the standard access tier. Also, remember that I’m estimating when my storage reaches 100GB here I’ll be billed AU$25.95 per month but until I reach that capacity and the less operations I do on files there, the cheaper this storage will be. I therefore expect my ‘real world’ costs to in fact be much less than this AU$25.95 figure over time.

Let’s now look at the next two storage locations, which will be Azure SMB file shares.

Unfortunately, the pricing calculator doesn’t allow us to easily calculate the price for an SMB Share on a cool access tier (Azure SMB files doesn’t currently support being on the archive tier). However, the pricing is only an estimate, so I know if I place it on a cool access tier it will be cheaper anyway, so I’m going to keep it simple.

image

Thus, for reference:

The estimated total for 1TB of SMB file storage on the standard performance tier = AU$106.58 per month.

remembering that for the standard tier we need to take into account the cost of operations as shown.

and for Premium:

image

The estimated total for 1TB of SMB file storage on the premium performance tier = AU$348.00 per month.

With premium storage, you don’t need to worry about operations, however don’t forget, if you go premium you’ll be paying for the total allocated capacity no matter how much you are actually using. Thus, I’ll again be sticking with standard storage.

So, for my 50GB Azure SMB files hot tier I calculate the following:

image

The estimated total for my 50GB of hot SMB file storage on the standard performance tier = AU$32.40 per month.

Now how can I get an idea of what the cool SMB file price will be? Although it is not this simple, I’m going to use a ratio from:

Azure Block blob pricing

image

So, by my super rough rule of thumb maths I get:

cool/hot = 0.02060/0.0275 = 0.75

Thus, cool storage is 75% the cost of hot storage say.

The estimated total for my 100GB of cool SMB file storage on the standard performance tier = AU$32.40 per month x 2 x 0.75 = AU$48.60 per month

The 2 x is because the hot price I have is only for 50GB and I want 100GB of cool storage.

In summary then, I will create 3 x storage repositories for my data:

– 100GB blob archive storage = AU$25.95 per month

– 100GB SMB file cool storage = AU$48.60 per month

– 50GB SMB file hot storage = AU$32.40 per month

250GB total storage estimated cost = AU$106.95 per month

Again remember, this is my estimated MAXIMUM cost, I expect it to be much lower until the data capacities actually reach these levels.

Now that I have the costs, how do I actually go about using these storage locations?

Because archive storage is blob storage I’ll need to access it via something like Azure Storage Explorer as I can’t easily use Windows Explorer. I’m not expecting all users to work with this data so Azure Storage Explorer will work fine to upload and manipulate data if needed by a select few.

As for the SMB file cool and hot storage I’m going to map these to two drives across my VPN as I have detailed previously:

Azure file storage private endpoints

This means they’ll just appear as drive letter on workstations and I can copy data up there from anything local, like a file server. The great thing is that these Azure SMB file shares are only available across the VPN and not directly from elsewhere as the article shows. That can be changed if desired, but for now that’s they way I’ll leave it. I can also potentially get to these locations via Azure Storage Explorer if I need to. The flexibility of the cloud.

So far we now have:

– Site to Site VPN to Azure (<5GB egress from/unlimited ingress to Azure)= $36.08 per month

– 100GB blob archive storage = AU$25.95 per month

– 100GB SMB file cool storage (mapped to Y: Drive) = AU$48.60 per month

– 50GB SMB file hot storage (mapped to Z: Drive) = AU$32.40 per month

Total maximum infrastructure cost to date = AU$143.03 per month

So we now have in place the ability to start shifting data that doesn’t make sense going into Microsoft 365 SharePoint, Teams and OneDrive for Business. Each of the three new storage locations has their advantages and disadvantages. That is why I created them all, to give me the maximum flexibility at the minimum cost

We continue to build from here in upcoming articles. Stay tuned.

My Apps – 2020

I am still not a big app user. I am very careful and selective about what I install on my device. Less is definitely more for me.

To see what I was using at the beginning of last year check out the article:

My Apps – 2019

Since this time last year, the biggest change has been moving to an iPhone as my primary phone early in 2019. I am sticking with the Apple ecosystem for at least 12 months to see what it is like. My experience so far is that it is functional but overall not as good as the experience on Android. The interesting thing towards the end of 2020 will be the new Android based Neo and Duo Surface devices that Microsoft have promised. The Duo device will also allow calls, so I’m thinking that I’ll maintain the iPhone as the primary device until the Duo becomes available. That may prove to be 2021 here in Australia, but I thinking that will probably be my next primary device change We’ll see.

My most used apps on mobile devices over the last year were:

Apple podcasts – Was my main podcast app until a reader recommended Castro on iOS. We’ll see how it goes, but it’s gotta be better than Apple podcast! I really miss Podcast Addict, which unfortunately is only available on Android

Lastpass password manager and authenticator. Google authenticator has gotten the flick as part of my limiting what Google apps I use.

Microsoft Authenticator – I use this for a number of select web sites as well as Microsoft 365.

Car Play – Connects to my daily drive to provide the ability to listen to podcasts as well as use Waze for navigation. Gotta say that it isn’t nearly as good as Android auto in my experience. However, since I’m spending an extended time in the Appel ecosystem I’ll be stick with this.

OneNote – is a must on every device I own. Syncs all my notes to every device. Allows me to not only truly have my information everywhere I am but also capture information quickly and easily.

OneDrive – This mobile app now not only allows me to manage my Microsoft 365 files but it also incorporates the more advanced Office Lens technology that scans and uploads, documents, whiteboards, etc.

Tripview – One of the few apps that I have happily paid for. I use this to let me know the Sydney train schedule to help me get around when I need to negotiate the ‘real world’.

Audible – If I can’t read my Kindle then I can normally always listen. This app allows me to listen to my audio books where ever I am.

Amazon Kindle – If I don’t have access to my Kindle then I can still read my books. In my case that will most likely be on my iPad. I also use the Kindle app on the iPad when the ebook has a lot of images that sometime don’t display well or are too small for the Kindle device.

The following as currently only iOS:

Oak – For mindfulness, breathing and meditation

Rode Reporter – which I use for recording many of my presentations when I am out on the road.

Of course I have all the social media apps, such as Twitter, Linkedin and Facebook on my devices.

I also have all the Microsoft/Office 365 apps. The ones I use the most are probably To-Do, Outlook, SharePoint, OneDrive, Teams and Yammer, although Word and Excel also get used regularly. Just about every Microsoft Office 365 service has an app that you should have on your mobile device. On my Android I am also using Edge as the primary browser along with the new Edge Insider.

I’ve also added the Intune app to all my devices so they can be better managed.

I use the Microsoft Next Lock Screen on my Android device.

Some occasional ones I use include:

Get Pocket

Duolingo

I use the normal personal apps for things like Internet banking and so on. I also use Blockfolio for monitoring cryptocurrency. For casual entertainment and general interest I also have Minecraft Earth installed.

One my iPad, which also serves as a personal entertainment device, I have the streaming services Netflix and Amazon Prime Video.

The above are my used apps across my various mobile devices. My aim to try and keep the app standard across all the devices and as few as possible. I try and standardise as much as possible to use the Microsoft apps on all platforms. I certainly use a wide variety of apps on my devices by prefer the desktop versions if available simply because my finger are too fat and my patience too short to be productive for long stints on mobile devices. My kingdom, my kingdom for a full keyboard and screen I cry.

Optimising Azure OMS data ingestion

image

Every month when I receive my Azure bill I take a careful look at it to see if there is anything I can optimise. This month I saw that the top cost was from my Log analytics workspace as you can see above. This however was no surprise because it basically represents that amount of data that had been ingested from my remote workstations into Azure Sentinel for analysis.

image

When I looked at Azure Sentinel I can see that I am bringing in more performance logs than security events per day. Now the question is, am I really getting value from having that much ingestion of performance logging? Probably not, so I want to go and turn it down a notch and not ingest quite so much and hopefully, save me a few dollars.

image

To do this, I’ll need to log into the Azure Portal and then go to Log Analytics workspaces.

image

I’ll then need to select Advanced settings from the menu on the left.

image

First thing I checked was in Data, Windows Event Logs is that I’m only capturing the errors in the Application and System logs for the devices, which I was.

image

Next I went to Windows Performance Counters and adjusted the sample time limit. I have increased it to every 10 minutes for now to see what difference that makes. I could also remove or add certain performance counters here if I wanted but I wanted to work with the current baseline.

With all that done, I’ll wait and see what the cost differences are in next month’s invoice and adjust again if necessary.