Using Retention Policies in Office 365

Before we get into this article I need to reinforce the following:

Retention is NOT the same as backup

Thus, what I am going to cover here should NOT be considered as a replacement to any existing backup policy you have for Office 365. What I’ll cover here is retention of data based on policies you set. Retention can be a way to preserve data as well as delete data based on a set of defined rules. You should consider retention policies as part of your compliance strategy not as part of the disaster recovery strategy.

The great thing about retention policies in Office 365 is that they are generally available across all plans. So what I detail here should apply to all Office 365 tenants.

Office 365 has no retention policies in place by default. This means that any existing data has no additional protection. Importantly, this means that existing data will NOT be covered by the policy UNTIL the data has been changed. Thus, if you create a retention policy and then go and delete data BEFORE making any changes to it, the data will NOT be saved! Once in place, the policy ONLY applies to data that gets altered (i.e. updated or modified) from that point on.

With that in mind the first step in the process is to create a retention policy. You do this by navigating to the Security and Compliance center in Office 365. From there, select the Data Governance option from the menu on the left and then Retention from the submenu as shown above. You should see that there no policies in place yet.

To create a new policy select the Create button on the right hand side of the screen.

Give your new policy a name and description and press the Next button at the bottom of the screen.

Here is where you need to decide what rules your policy will have. In this case I have chosen to retain data for 7 years based on when it was created and to not delete it after this period.

You’ll note that you can create policies that also delete data so be very careful when you select those options.

The bottom of the page allows you to use more advanced retention settings. In here there should two options to select from as shown above.

The first option allows you to apply the policy via keyword or phrase. You simply enter those terms into the editor that is displayed when you select the option.

Once you have entered the keywords you wish, you’ll need to enter the standard retention options as shown above.

The second advanced retention option allows you to apply the policy based on ‘sensitive information’. As you can see from the above, you can select from a range of pre-configured sensitive information types that can be scoped to your country. Here, I am selecting Australian Financial Data.

If you look at the policy you will see what information it consider ‘sensitive’. In this case, the policy will match things like Australian SWIFT banking codes, Tax File Numbers, Bank Accounts and Credit cards.

Once you have set the data types for your policy, you’ll need to nominate which locations inside Office 365 this retention policy will apply to. You can apply the policy across all or specific data inside Office 365 as shown above.

You’ll see that you can target Exchange mail, SharePoint Online,

Groups (as well as Teams), Skype and Exchange public folders.

You’ll see that you can also include an/or exclude specific locations inside each service if you wish. Simply select the Choose hyperlink and make your selections as shown above.

Once you have completed all these options you can then Create this policy and apply it immediately or Save for later application.

In this case I’ll create the policy and apply it immediately. Note the message at the top of of the dialog that tells you it may take a full day for the policy to be applied. I would suggest that you do wait a full day for the policy to be applied throughout your tenant before you continue.

After creating the policy you will see that the Status is On but it is Pending as shown above.

If you select the information icon you’ll see that what you want to wait for is the On (Success) option to be displayed here.

After waiting a suitable amount of time and checking the policy status you will find that it has succeeded as shown above.

At this point the policy is in place and is protecting any data that is now changed.

With the retention policy in place let’s go to the location of some file data in a SharePoint Team Site, specifically a Document Library as shown above.

Before we do anything, let’s check out that the Site actually contains.

We see that there is nothing special as yet. There will be, just not yet.

The retention policy will only act on changed documents from the point it was enabled. So we select a document in the library and edit it.

The document is changed and saved back to the library.

Now the file is still in it’s original location and the retention policy is applied. As the original file still existing in its original location the retention policy doesn’t need to take any action.

However, if the original file is now deleted from its original location as shown above what will happen?

Any document deleted from a SharePoint Document Library is sent to the Recycle Bin.

If we look in Recycle Bin we see the deleted document as shown again. The retention policy still does not yet need to take any actions as the document is still available, however remember, that items don’t stay in the SharePoint Recycle Bin forever. They are aged out after a total of 93 days. Thus, the retention policy doesn’t need to do anything until this time period is exceeded.

However, it is also possible for the user to delete the file from their recycle bin as shown above.

Once the user has deleted the file from their recycle bin the file will move to an administrator recycle bin or the remainder of the 93 days. Again, the retention policy doesn’t need to take any actions until this time period is exceeded.

At the point at which the file is going to be purged from the Office 365 environment the retention policy that was configured kicks in. It creates a new document library in the Team Site called Preservation Hold Library as shown above.

This new document library is only available for administrators to view and when you look in here you will see all versions of the deleted file. Remember, that every time you change a file in SharePoint it create a previous copy.

Thus, as an administrator, we can recover a file from this location for the period of the retention policy, which in this case is 7 years. Once the conditions of the retention policy no longer apply to the file (here it is > 7 years) the file will be removed permanently within 7 days from the tenant.

You can find lots more information about Office 365 retention policies here:

Overview of retention policies

In there, you will note for email data:

To include an Exchange Online mailbox in a retention policy, the mailbox must be assigned an Exchange Online Plan 2 license. If a mailbox is assigned an Exchange Online Plan 1 license, you would have to assign it a separate Exchange Online Archiving license to include it in a retention policy.

So, retention policies are a good way to manage the compliance of your data. As I said at the start, they are NOT a replacement for backup, however they do provide an extra layer of protection for you information and can be implemented quite easily as you can see above.

The last thing to remember is that retained data has to live somewhere and will consume you tenant space availability across the difference services. The more locations and data protect, the more copies of previous data you will have. So keep it simply and limit what you want to retain. This means planning your retention strategy in advanced rather than bulk applying it to all data in all locations.

Finally, remember that retention policies are available across the range of Office 365 license and I would encourage you to take advantage of them.

Offline file conflicts with SharePoint Online

It has been over three years since I wrote an article about file conflicts in Office 365 –

Resolving OneDrive for Business file conflicts

and as you can appreciate a lot has changed since then. Probably the biggest change is that we now have File on Demand and the ability to sync SharePoint Document Libraries. However, there will always remain challenges around shared files going offline when multiple people continue to work on them.

I will preface all this by saying that it is best practice to ‘Check Out’ any files you wish to use prior to you going offline. Doing so will ensure you have exclusive write access to that file while you are offline and until you check that file back in.

Of course, not everyone is going to follow best practice and we are going to end up with the following scenario.

Let’s say that Lewis Collins (user 1) creates a new Excel spreadsheet called conflicts.xlsx in a SharePoint Document Library as shown above.

If Lewis opens that file using Excel Online and makes a change by adding the entry ‘Online 2’, as shown above, it is automatically saved back to the SharePoint Online Document Library.

A second user (Robert Crane – user 2) used OneDrive Files on Demand to sync a copy of that same file to their desktop as shown above.

This second user (user 2) now opens the file using Excel on desktop and makes changes to the file by adding the entry ‘Offline 3’ as shown.

You can see that because the user is still connected to the Internet any changes are automatically synced back to the SharePoint Online Document Library.

So, while everyone is online all changes are updated into the one location.

We can also look at the version history of the file and see all previous versions thanks to automatic version history in SharePoint Document Libraries. We can roll back or view any of these if we wish.

At this point, user 2 (Robert Crane), goes offline and is no longer connected to the Internet.

Now because user 2 didn’t check the file out prior to going offline, user 1 can continue to edit the file. They do so adding the entry ‘Online 4’ to the file, which is then immediately saved back to the SharePoint Document Library.

While offline, user 2 adds a new entry to their offline version of the same file. Here they create an entry ‘Offline 4’ as shown above.

Thus, we now have a situation where the file in SharePoint Online is different from the file on the users desktop. This will clearly create a conflict when user 2 return online.

User 2 comes back online and at the next sync is informed of a conflict as noted in their file manager as shown above.

When user 2 attempts to open the file in conflict they are presented with the warning banner at the top as shown. They are given the option to either Save a Copy or Discard Changes.

If they select Discard Changes, any updates they have made to the file while they have been offline will be overwritten with what is currently in SharePoint Online. Once they select this, any updates they have made to the file while they are offline will be lost and the copy they have on their desktop will be the same as what is currently in SharePoint Online. In short, their local copy is overwritten with that from SharePoint Online. They can’t recover their original file after this happen because the file they changed was only saved to their desktop.

If they select Save a Copy, the file they have changed will be uploaded to SharePoint Online replacing the current version in SharePoint Online.

The OneDrive sync client will then kick in and copy the file from user 2’s desktop to SharePoint Online Document Library replacing the version that others have been working on and potentially removing changes they have made.

When the sync is complete, user 2 should see the same situation on their desktop, as shown above, prior to going offline.

Now, the file that was changed by user 2 while they were offline has become the primary file in SharePoint Online and on desktops. However, any changes that user 1 made while user 2 was offline are no longer in the most current version of the file.

Before we tackle that situation let’s look another experience for user 2 as they come back online with a different version of the file.

When user 2 comes back online with a different version of a file they will also see the system tray icon for their sync client display a warning as shown above.

If they select this the sync client will open and display a conflict message as shown above.

Clicking that message will show them greater detail on the conflict as shown above.

If they click to resolve the issue they will be presented with the above dialog providing two options.

The option Open in Office to merge changes will simply open Excel and take the user through the experience detailed above, i.e. save a copy or discard changes.

The second option Keep both files will rename the changed version on the desktop to conflicts-.xlsx. Thus, the original file they were working on offline will be renamed and the newer version that is in SharePoint Online will be downloaded to the original name on their desktop. The idea is basically to create a second copy of the file, rather than overwriting the original. Users would then need to open both files and manually merge any changes back to a single file. The end result here is two files with different names, each holding the unique changes made by each user.

Let’s return to the situation where user 2, who was offline, comes back online, opens the file in conflict and selects to save their copy back into SharePoint Online by using the Save a Copy button.

This means that any changes user 1 made to the file while user was offline are ‘lost’ because user 2 has overwritten the file with their version.

However, don’t forget that SharePoint Online Document Libraries include automatic versioning. This means that when user 2 uploaded their file, the file user 1 had been working on isn’t deleted, it is simply saved as a previous version. So, both files are still in SharePoint Online in full fidelity. One is current and one is the previous version.

You have the ability to compare previous versions or restore previous versions if you wish.

My experience is that Excel is a fairly complex program and in most cases you’ll have to manually merge any changes between the two documents. However, as you can see above, with Word the application can generally merge changes automatically for you using the revisions ability built into the program.

As I said at the beginning of this article, best practice is to check document out prior to going offline to avoid conflicts. If that doesn’t transpire, then you probably need to manually merge changes using versions in SharePoint Online. However, as you can hopefully see SharePoint Online will retain both versions of the file if you do go offline. I would suggest however, you have a play with exactly how this works in your environment prior to requiring it. SharePoint is magic but it doesn’t read minds, yet!

Free legacy SharePoint Documentation and training

About 3 years ago I opened sourced all my SharePoint on premises documentation that used to be part of my Windows SharePoint Operations (WSSOPS) Guide as I details here:

https://blog.ciaops.com/2015/01/free-sharepoint-documentation.html

and is made available under Creative Commons Attribution-Non Commercial-ShareAlike-4.0 Internation license. In essence means it free to distribute but can’t be resold.

Initially it lived on my web site, then I moved to Docs.com. Unfortunately, Docs.com has now been fully retired so I need to find a new home for this.

I the decided to move everything to the CIAOPS Academy.

2023 update – I have moved the content into my GitHub repo at:

https://github.com/directorcia/general/tree/master/Archive/WSSOPS

All the information remains free to access and download but thanks to the platform I now use I can start adding additional training material, like my YouTube videos, into the curriculum hopefully adding some more value.

Please remember, that all the information here is provided ‘as is’ and is no longer maintained. It remains free to download and re-distribute, so if you want to put it somewhere else on the Internet, be my guest. However, remember it can’t be changed if you do and I’d also appreciate a heads up on where you have placed it just so I can monitor any comments or feedback.

I’ll keep adding to what’s up there but it isn’t a priority so please use it of you need to and let other know who still may require this information.

Post News directly from SharePoint summary page

When you go to your SharePoint summary page you’ll see all your SharePoint Teams Sites as well as a list of news articles bubbled up to the page from these locations.

If you now look closely you’ll see (or soon see) a link, Create news post, at the top of this page as shown above.

When you select the link you’ll be prompted for a location you have access to in which to post the article. Simply select the location you want to the article to actually live in.

Now go and create the news article as you normally would.

Although this change may seem seem small it reduces the friction for people when they want to publish information. The faster they can do this the better and more likely they are to share something with everyone.

Making changes to existing Site Collections

Here’s a quick summary video on how to make changes to a traditional SharePoint Site Collection. This will only work with pure SharePoint Teams Sites, not those created with Microsoft Teams and Office 365 Groups. That will become available in the new SharePoint Admin Center coming soon. Till then, this video will show you how to access the settings as well as how to interact with the most common configurations.

SharePoint Online Social and Newsfeed Basics

Here’s a quick tutorial to show you how to display Team Sites on the SharePoint Online summary page using the ‘Follow’ option. You’ll also see how the Team Site Newsfeed works, how to create new stories and where the information is saved in a Team Site.

Basic Calendaring in SharePoint Online

Here’s a short video tutorial on the basics of using a calendar in a SharePoint Team Site. You’ll see how to add a calendar to a Site as well as add it to the navigation and create events.