A framework for file migrations to Microsoft 365

One of the major points of confusion and poor execution I still see today is the approach that many take to migrating files to Microsoft 365 and Office 365.

Many years ago I wrote a number of articles about this:

The classic SharePoint migration mistake

SharePoint Online – Pilers and Filers

SharePoint Online migration – Start up is key

and all of that is still valid and I recommend you read it, however the technology has moved on somewhat and it is now perhaps time for an update.

What I’ll cover here is a framework for migrating on premises data, typically on file servers in network shares, into the collaboration tools in Microsoft 365. And that’s the first point. You need to look at this across all the services Microsoft 365 provides you today. Not just SharePoint. Not just OneDrive. And not just Teams. Microsoft provides a range of services that you should consider in this file migration process.

Now many claim that because of all these options it is too complex and therefore you should just dump your data in one place. I can’t tell you how many times I’ve see exactly this, everything from a file server F: drive dumped into a single Document Library in SharePoint. Oh the pain.

image

The secret of success is to have a “system” and not do things randomly without thought. Thus, my framework is shown above. Now let me break down the pieces for you.

image

You firstly start with the source (on the left i.e. a file server drive) and the destinations (on the right, i.e. collaboration services in Microsoft 365).

image

The first step is to filter the source information and remove (yes I said DELETE) stuff that doesn’t make sense to move. Old and duplicate files are example but I have highlighted this in more depth in a previous article:

Data discovery done right

Thus, you should now have less data to move because you have thrown some away. If you haven’t then you are not being serious about this. You should also have a better idea about what data to archive, what is user data, what is common data and so on.

image

With the remaining good data you move (yes, I said move NOT copy) users personal data to their OneDrive. You actually get them to do that so firstly they get familiar with the new environment, they move only what they want to keep and you crowd source this task reducing the workload. We all know that some people will never ‘get around’ to moving their data, so after a suitable time period has elapsed you move it for them. The typically data that is moved in this process is anything on the desktop, home drive or in user profiles.

image

Next, is to identify the common data by function or location and move it to a Microsoft Team. In a typical on premises file server the existing shares you will find data stored in folders like F:\Finance and F:\Administration and so on. These top level shares are most likely going to be the name of the Team and the next level down folders to be the channels inside that Team. In many cases, users may also wish to ‘clean up’ the existing or ‘start fresh’, which is also fine. However, it is relatively easy to find the data that should go into a Microsoft Team and move it there.

image

The next data to move is common data that makes sense living in it’s own dedicated SharePoint Team site. Data that goes into stand alone SharePoint is data that does not require chat and conversations around it, Teams is where that data would live.

The best example here is probably Archive data. This is data the business want to keep ‘just in case’ but won’t be updated by users. Thus, you move the archive data as is, into it’s own designated SharePoint Team site, mark it as read only for everyone so they can use it as a starting point if they need to. The major advantage of moving the archive data to SharePoint is that it is now searchable using the tools in Microsoft 365.

Another good example of a stand alone SharePoint Team site would be an Extranet from which you want external parties to come and download data. Having a stand alone SharePoint Team site makes managing securities much easier.

image

The data that is now left is typically company wide data like policies, procedures, manuals, etc. This is then moved into the traditional ‘Intranet’ using a nice pretty SharePoint Communications Site. This Intranet is available to all and typically used to consume data i.e. get something, read something. New data for this Intranet comes from the output of the individual Microsoft Teams created earlier. For example, imagine the Finance team produces an annual budget. They do this creation inside their own Team and publish the finished result into the Intranet for everyone in the business to consume. I’ve covered this concept in greater detail in a previous article:

The layers of Office 365 collaboration

image

Now you have data all over the place inside Microsoft 365 as many will point out correctly. What’s missing is a consistent method of navigation between all these sources to make it easy for users to navigate. This is where SharePoint Hub Sites come in. You uses this to provide a consistent navigation over the top of all your data. To users this makes it all appear very logical and structured (even perhaps like their old file server) and yet provides the flexibility to be reconfigured at any stage and have those changes automatically applied across the whole structure thanks to SharePoint Hub sites.

image

The final layer is Yammer. This is for company wide communications such as water cooler chat (birthdays, sports, holidays), information on how to better use Microsoft 365, questions about the business, messages from CEO, suggestion boxes, what’s happening with the business or competition, sales, wins and so on.

Like the Intranet, Yammer is designed for corporate wide chat that doesn’t fit into any specific Team (which are arranged by function or location). I have detailed the importance of Yammer in a previous article:

Why Yammer is still relevant

image

Now that you have a linear framework that builds on top of itself, hopefully you have a better picture of where you can put data from a file server into Microsoft 365. Having a framework is great but more importantly is how to get users to also adapt to working in this way. My approach is to start with Yammer and OneDrive first as I have detailed previously:

Focus on the ‘Me’ services first

In short, if users get used to working with just files using OneDrive and then with just chat in Yammer, they will find it much easier to then move to Teams which is effectively files and chat combined. This thinking stems from:

The rule of three

I advocate to not over load people. This is because you need to:

Stop making your users feel stupid

You can have the greatest technology in the world but if no one uses it then it is pretty much wasted. that’s what I see with Microsoft 365. Not enough through and time is devoted to adoption and transformation. Most migrations are done as fast as possible to ‘close the ticket’ and move onto the next. The fallacy of that rushed approach is evident in failed adoption and user frustration.

So there you have it, a framework for migration of file data into the world of Microsoft 365. You can use this framework in whatever way makes sense to you. You can use all of it, none of it or as a basis for your own approach. However, the key message is to HAVE a process rather than something random. Remember, moving to a world of collaboration from a world of storage requires transformation not just migration!

Once you have the top level framework or process, you can then start developing individual frameworks and adoption processes for each piece. That is, one for OneDrive, one for Teams, One for Yammer and so on. Once you have these processes, next you can start automating them to improve reliability and allow you to scale. Without a system you can’t automate. Without automation your effectiveness and profitability continue to fall, so investing the time in a collaborate system is definitely worthwhile. It makes users happier and it makes IT more profitable! A win – win. That is what a move to Microsoft 365 should be all about!

Unable to save attachments to SharePoint Online

One of the most important things when you implement adoption is to have a positive initial experience. This typically means ‘easing’ a user’s transition during the adoption process. If too many things are different, then there is much more likely to be a negative impression of the new processes. This slows adoption and at worst, can actually halt it in its tracks.

When moving to Microsoft 365, one of the most common things that a user needs to accomplish to be able to save and add attachments to emails. They have been performing this seamlessly using on premises file servers for years. They simply select to attach and then navigate to the file, attach it, then send. Easy.

Unfortunately, as I have documented before:

Saving attachments to SharePoint

it isn’t easily done with SharePoint Online. This is really strange, given that SharePoint Online is the place where users should save and access common files in the Microsoft Cloud. Let’s take a look at the issues I’m taking about.

image

So an email arrives in my inbox on Outlook on the desktop, as shown above.

image

I want to upload this directly into an existing SharePoint stand alone Team Site, but as you can see the only option I have is my own personal OneDrive for Business or a range of Office 365 Groups and Teams that already exist.

Just to make sure I haven’t missed anything, I’ll select the More option at the bottom of the list.

image

Now I only have the option to save to a Group (which includes Microsoft Teams). So, let’s say I select the Sales Group (which is actually a Microsoft Team).

I’m now returned to Outlook. Where did that attachment actually go?

image

So, if I call up my Sales Team and rifle through all the file locations in Teams interface, I can’t find the file as you see above!

image

Turns out that the attachment I saved is placed into the root of the default Document Library in the Microsoft Team as you see above. But guess what? There is no way to actually see that unless I navigate to that location via SharePoint. I actually can’t see that attachment I just saved if I’m using the Microsoft Teams app! They all end up in the root of the Documents location, which isn’t accessible in the Teams app!

image

This means, that the only REAL solution for users to save the document to other locations in various SharePoint Document Libraries, is to firstly sync those destination locations to their desktop and then save the attachments the old fashioned way to the sync location so they will end up in SharePoint.

That means, to save or add attachments I firstly have to sync EVERY location I might want to save a file too!

image

Outlook Web Access is actually worse than the desktop client as the only options you have are to download or save to OneDrive for Business as seen above.

image

Interestingly, if I want to attach a file from a SharePoint site I can navigate to Browse Web Locations, select the Team Site I want

image

and I see a Windows Explorer pane where I can navigate to locate the file I wish to attach, just like on premises days. However, the look and feel here is pretty dated and requires Windows Explorer to be working and may pop up warning dialogs which will freak most users out.

image

When I use Outlook Web Access I can Browse cloud locations for an attachment

image

I effectively only see my OneDrive for Business as shown above.

These experiences leave a bad taste in the mouth for users, especially first time users grasping with the ‘modern’ way of working. They need to have an experience which is pretty much identical to the one they had on premises. Why can’t we simply save and add attachments directly from SharePoint Online Team Sites like we have always been able to do from on premises network file shares?

I’m seeing this end user frustration more and more in the field and was prompted to write the article to hopefully rally the masses to get a change enacted. So the best thing you can do is visit this UserVoice request:

https://office365.uservoice.com/forums/264636-general/suggestions/18553747-please-enable-the-attachment-of-sharepoint-files-w

and vote it up.

Next, tweet about getting this enabled to the following accounts:

https://twitter.com/Outlook

https://twitter.com/SharePoint

https://twitter.com/Microsoft365

and

https://twitter.com/jeffteper

I will be!

Perhaps I’m missing something obvious here and if I am please let me know but I don’t think I am. Help me raise awareness and improve Outlook so it is easier for users to adopt Microsoft 365!

Preventing Malware downloads from Office 365

image

If you are unfortunate enough to somehow get malware in your Office 365 tenant you may not appreciate that by default you can still download this, even though it gets detected as shown above.

image

Best practice would be to use the PowerShell command:

Set-SPOTenant –DisallowInfectedFileDownload $true

to prevent users from having the option to download the infected file. Basically, it removes the Download button as shown above. Doing this will apply the setting across all SharePoint Sites, including OneDrive for Business, Teams and stand alone site collections.

From the Microsoft documentation:

If the Set-SPOTenant cmdlet has the DisallowInfectedFileDownload parameter set to:

true (recommended), this happens:

  • All actions, except Delete, are blocked for detected files.

  • People cannot open, move, copy, or share detected files.

  • People see a visual cue that indicates that a file has been identified as malicious. No one can download the file.

false, this happens:

  • All actions, except Delete and Download, are blocked for detected files.

  • People cannot open, move, copy, or share detected files.

  • People see a visual cue that indicates a file has been identified as malicious, but they can choose to accept the risk and download the file anyway.

Allow up to 30 minutes for your changes to spread to all Office 365 datacenters.

The recommended best practice is then to turn this on for all tenants as it is not on by default.

Azure AD and SharePoint Online user differences

I’ve been developing scripts to work with OneDrive for Business when I fell into a bit of a rabbit hole that lead me to an interesting revelation.

Part of the challenge with working with OneDrive for Business in Office 365 is that not all users have one, even though they are licenses for it. The reason for this is simply that a user’s OneDrive for Business isn’t generally provisioned for them until they start using it. Thus, in my demo tenant there are probably users who haven’t as yet been through the process of having a OneDrive provisioned. No issues.

Secondly, when you share information with external users in SharePoint and Teams you may also find an AD account but that user hasn’t as yet access SharePoint resources for some reason. Maybe, they haven’t accepted the sharing request and so on. Again, no big deal.

image

So I created a script that goes through each active Azure AD user in the Office 365 tenant and check to see whether there is a corresponding SharePoint Online user. To do this I used the following commands:

get-spouser

and

get-azureaduser

So I trained these commands on the OneDrive for Business URL which is typically:

https://tenantname-my.sharepoint.com

As you can see from the above report, the green lines indicates matches to accounts in my Azure AD and in my OneDrive for Business. The green tenant users, with a custom domain typically have their own OneDrive for Business. The green External users, distinguished by an account that includes #EXT# are typically accounts outside the tenant that have been shared information with and accepted that sharing request.

Now the red tenant users, typically haven’t had their OneDrive for Business provisioned yet and the red external users typically haven’t accepted the sharing request that has been sent them as yet. All understood.

image

Here’s where the rabbit hole opened up. Ok, I thought, now what happens if I do the reverse? That is, check my SharePoint users against my Azure AD users? So off I went to create a script.

The script came back with the results you see above. All the the yellow accounts are SharePoint users that don’t have a match Azure AD account. Quite a few eh? When I first saw this I panicked a bit, because many of the accounts I didn’t recognize. What was going on here I wondered? Had I been compromised?

In a perfect world, there would be a one to one mapping between Azure AD accounts and SharePoint account. However, things aren’t that perfect, so in my demo tenant, I had created lots and lots of accounts over the years and many had become ‘orphaned’ leaving behind information in SharePoint. Many were just so old I forgotten that I created them and then later deleted the Azure AD account.

Is this a problem? Not really I don’t think, because without an Azure AD account to login to, these ‘orphaned’ resources aren’t much use. Still, if they aren’t needed then they really should be deleted to my mind.

Interestingly, some of these ‘orphaned’ SharePoint users actually still had their own OneDrive for Business that clearly wasn’t being displayed anywhere else. Once I took control of these ‘orphaned’ sites by making myself a Site Collection Administrator I could see what they actually contained. When I was happy it wasn’t needed or in use I deleted these, again using PowerShell.

So what did my trip down the rabbit hole teach me? Firstly, I learned that Azure AD and SharePoint user accounts don’t always line up. Next, I learned that you can end up with ‘orphaned’ SharePoint users and resources that you may want to clean up using PowerShell. I don’t believe these represent any security issues but if they aren’t necessary then they probably should be deleted. However, be careful of system accounts which shouldn’t be removed. Just get rid of those you recognise as no longer being required.

The biggest thing that my exploration taught me is the value of PowerShell to get behind the standard interface of Office 365 and see what is really going on. It gives you much better control and for me it helps me understand much better how everything works.

If you want the scripts that I used to do these comparisons then I suggest you sign up to my Patron community – www.ciaopspatron.com where you’ll find these and whole lot more Office 365, Microsoft 365 and Azure resources.

Pssst…want some free GBs in your OneDrive for Business?

One of the common beliefs with Office 365 is that OneDrive for Business storage for most plans (typically Business plans) is limited to 1TB per user. Well, I’m here to tell you that the limit for most tenants is in fact 5TB per user. Don’t believe me? Well, read on and be AMAZED!

image

You can see from the above that the user has the standard 1TB storage for the OneDrive for Business.

image

The ‘normal’ way that you set the amount of storage each user gets for their OneDrive for Business is via the Storage option in the OneDrive Admin console as you can see above.

Now, if you visit the link just below that setting you will see the following:

image

Here’s the full link:

https://support.office.com/en-us/article/set-the-default-storage-space-for-onedrive-users-cec51d07-d7e0-42a3-b794-9c00ad0f0083?ui=en-US&rs=en-AU&ad=AU

Thus, if you have more than 5 users (and perhaps less) you can get 5TB per user OneDrive for Business.

image

These days, I prefer to do most of my administration using PowerShell. The above script will set the new limit for all users provisioned with OneDrive for Business from this point on to have 5TB of space in their OneDrive for Business.

image

To increase any existing users OneDrive for Business up to the 5TB limit you’ll need to run the above script for each user. You’ll need to replace the URL with each users individual OneDrive for Business URL.

image

After doing this, if you now look at the users OneDrive for Business storage quota, you’ll see it is now 5TB!

Magic eh? And you thought I couldn’t give you free GB’s out of thin air! Shame on you.

OneDrive Office sync conflicts

I recently wrote an article about

Offline file conflicts with SharePoint Online

that ran through the process of what happens when users go offline when working on shared files.

After doing some more poking around in the latest OneDrive for Business sync client I found this under the Office tab in Settings:

image

You can find more information on the first option here:

Use Office 2016 to sync Office files that I open

which notes:

If you turn off this setting, Office will no longer be able to automatically merge changes from different versions of documents. You’ll also be prompted to upload a new copy of a file before you can share it directly from an Office desktop app.

You can also elect how to handle Sync conflicts, which by default is set to Let me choose to merge changes or keep both copies.

The defaults options are going to suit most people but you can go in a customise these if you wish to improve how conflicts are handled in your environment.

Offline file conflicts with SharePoint Online

It has been over three years since I wrote an article about file conflicts in Office 365 –

Resolving OneDrive for Business file conflicts

and as you can appreciate a lot has changed since then. Probably the biggest change is that we now have File on Demand and the ability to sync SharePoint Document Libraries. However, there will always remain challenges around shared files going offline when multiple people continue to work on them.

I will preface all this by saying that it is best practice to ‘Check Out’ any files you wish to use prior to you going offline. Doing so will ensure you have exclusive write access to that file while you are offline and until you check that file back in.

Of course, not everyone is going to follow best practice and we are going to end up with the following scenario.

image

Let’s say that Lewis Collins (user 1) creates a new Excel spreadsheet called conflicts.xlsx in a SharePoint Document Library as shown above.

image

If Lewis opens that file using Excel Online and makes a change by adding the entry ‘Online 2’, as shown above, it is automatically saved back to the SharePoint Online Document Library.

image

A second user (Robert Crane – user 2) used OneDrive Files on Demand to sync a copy of that same file to their desktop as shown above.

image

This second user (user 2) now opens the file using Excel on desktop and makes changes to the file by adding the entry ‘Offline 3’ as shown.

You can see that because the user is still connected to the Internet any changes are automatically synced back to the SharePoint Online Document Library.

So, while everyone is online all changes are updated into the one location.

image

We can also look at the version history of the file and see all previous versions thanks to automatic version history in SharePoint Document Libraries. We can roll back or view any of these if we wish.

At this point, user 2 (Robert Crane), goes offline and is no longer connected to the Internet.

image

Now because user 2 didn’t check the file out prior to going offline, user 1 can continue to edit the file. They do so adding the entry ‘Online 4’ to the file, which is then immediately saved back to the SharePoint Document Library.

image

While offline, user 2 adds a new entry to their offline version of the same file. Here they create an entry ‘Offline 4’ as shown above.

Thus, we now have a situation where the file in SharePoint Online is different from the file on the users desktop. This will clearly create a conflict when user 2 return online.

image

User 2 comes back online and at the next sync is informed of a conflict as noted in their file manager as shown above.

image

When user 2 attempts to open the file in conflict they are presented with the warning banner at the top as shown. They are given the option to either Save a Copy or Discard Changes.

If they select Discard Changes, any updates they have made to the file while they have been offline will be overwritten with what is currently in SharePoint Online. Once they select this, any updates they have made to the file while they are offline will be lost and the copy they have on their desktop will be the same as what is currently in SharePoint Online. In short, their local copy is overwritten with that from SharePoint Online. They can’t recover their original file after this happen because the file they changed was only saved to their desktop.

If they select Save a Copy, the file they have changed will be uploaded to SharePoint Online replacing the current version in SharePoint Online.

image

The OneDrive sync client will then kick in and copy the file from user 2’s desktop to SharePoint Online Document Library replacing the version that others have been working on and potentially removing changes they have made.

image

When the sync is complete, user 2 should see the same situation on their desktop, as shown above, prior to going offline.

Now, the file that was changed by user 2 while they were offline has become the primary file in SharePoint Online and on desktops. However, any changes that user 1 made while user 2 was offline are no longer in the most current version of the file.

Before we tackle that situation let’s look another experience for user 2 as they come back online with a different version of the file.

image

When user 2 comes back online with a different version of a file they will also see the system tray icon for their sync client display a warning as shown above.

image

If they select this the sync client will open and display a conflict message as shown above.

image

Clicking that message will show them greater detail on the conflict as shown above.

image

If they click to resolve the issue they will be presented with the above dialog providing two options.

The option Open in Office to merge changes will simply open Excel and take the user through the experience detailed above, i.e. save a copy or discard changes.

The second option Keep both files will rename the changed version on the desktop to conflicts-.xlsx. Thus, the original file they were working on offline will be renamed and the newer version that is in SharePoint Online will be downloaded to the original name on their desktop. The idea is basically to create a second copy of the file, rather than overwriting the original. Users would then need to open both files and manually merge any changes back to a single file. The end result here is two files with different names, each holding the unique changes made by each user.

image

Let’s return to the situation where user 2, who was offline, comes back online, opens the file in conflict and selects to save their copy back into SharePoint Online by using the Save a Copy button.

This means that any changes user 1 made to the file while user was offline are ‘lost’ because user 2 has overwritten the file with their version.

image

However, don’t forget that SharePoint Online Document Libraries include automatic versioning. This means that when user 2 uploaded their file, the file user 1 had been working on isn’t deleted, it is simply saved as a previous version. So, both files are still in SharePoint Online in full fidelity. One is current and one is the previous version.

image

You have the ability to compare previous versions or restore previous versions if you wish.

image

My experience is that Excel is a fairly complex program and in most cases you’ll have to manually merge any changes between the two documents. However, as you can see above, with Word the application can generally merge changes automatically for you using the revisions ability built into the program.

As I said at the beginning of this article, best practice is to check document out prior to going offline to avoid conflicts. If that doesn’t transpire, then you probably need to manually merge changes using versions in SharePoint Online. However, as you can hopefully see SharePoint Online will retain both versions of the file if you do go offline. I would suggest however, you have a play with exactly how this works in your environment prior to requiring it. SharePoint is magic but it doesn’t read minds, yet!