ASD Conditional Access policies comparison script

Screenshot 2025-11-26 092018

I have taken the ASD Conditional Access policy recommendations here:

https://blueprint.asd.gov.au/configuration/entra-id/protection/conditional-access/policies/

and created a script here:

https://github.com/directorcia/Office365/blob/master/asd-ca-get.ps1

that will compare your existing Conditional Access configuration to what the ASD recommends and tell you what you should consider changing to bring your policies more in alignment with those from the ASD.

Screenshot 2025-11-26 092225

Above, you’ll see one policy evaluation and recommendation outputted to a HTML file for easy reading.

The documentation for the script is here:

https://github.com/directorcia/Office365/wiki/ASD-Conditional-Access-Policy-Evaluation-Script

I look forward to hearing what you experience is using my script.

Implementing a Phased Rollout of Conditional Access Policies Requiring Device Compliance in Microsoft 365

Overview

Implementing Conditional Access policies requiring device compliance in Microsoft 365 requires careful planning and a phased approach to minimize disruption while maintaining security. This comprehensive guide provides step-by-step instructions specifically tailored for small businesses.

1. Prerequisites and Initial Setup

Required Licenses

  • Microsoft Entra ID P1 or P2 – Required for Conditional Access
  • Microsoft Intune – Required for device compliance management
  • Microsoft 365 Business Premium or higher for small businesses

Essential Preparations

  1. Configure Emergency Access Accounts
    • Create at least two emergency access (break-glass) accounts
    • Exclude these accounts from ALL Conditional Access policies
    • Store credentials securely and separately
  2. Create Device Compliance Policies First
    • Define minimum OS version requirements
    • Set encryption requirements
    • Configure password/PIN requirements
    • Establish jailbreak/root detection settings
  3. Enable User Registration for MFA
    • Allow users to register authentication methods before enforcing policies
    • Communicate registration requirements to all users

2. Phased Rollout Strategy

Phase 1: Foundation (Weeks 1-2)

Objective: Establish baseline security and prepare infrastructure

  1. Create policies in Report-Only Mode
  2. Block legacy authentication protocols
  3. Secure the MFA registration page
  4. Target privileged accounts first with phishing-resistant MFA

Phase 2: Pilot Testing (Weeks 2-4)

Objective: Test with limited user groups

Pilot Group Selection

  • Start with 5-10% of your organization
  • Include IT staff and willing early adopters
  • Avoid executives and VIPs initially
  • Ensure representation from different departments

Creating the Policy in Report-Only Mode

  1. Navigate to Microsoft Entra admin centerConditional AccessPolicies
  2. Create new policy with these settings:
    • Name: “Require Device Compliance – Pilot”
    • Users: Select pilot group
    • Cloud apps: Start with non-critical apps
    • Grant: Require device to be marked as compliant
    • Enable policy: Report-only

Phase 3: Gradual Expansion (Weeks 4-8)

Objective: Progressively include more users and applications

Automated Phased Rollout Approach

If using the Conditional Access Optimization Agent (requires Microsoft Security Copilot):

  1. The agent automatically creates a 5-phase rollout plan
  2. Groups are assigned based on risk and impact analysis
  3. Automatic progression between phases based on success metrics
  4. Built-in safeguards pause rollout if sign-in success rate drops below 90%

Manual Phased Rollout Approach

  1. Phase 3a: Add 25% more users (low-risk departments)
  2. Phase 3b: Add another 25% (medium-risk departments)
  3. Phase 3c: Add remaining standard users
  4. Phase 3d: Include executives and VIPs
  5. Phase 3e: Apply to all cloud applications

Phase 4: Full Deployment (Week 8+)

  1. Switch policy from Report-only to On
  2. Monitor for 2 weeks before removing report-only policies
  3. Clean up redundant or test policies

3. Monitoring Strategies

Real-Time Monitoring Tools

A. Sign-in Logs Analysis

  1. Navigate to Microsoft Entra admin centerMonitoring & healthSign-in logs
  2. Filter by:
    • Conditional Access status
    • Failure reasons
    • Affected users
  3. Review the Report-only tab for policy impact without enforcement

B. Conditional Access Insights Workbook

Requires Azure Monitor subscription:

  • Provides aggregate view of policy impacts
  • Identifies potential issues before enforcement
  • Shows user impact analysis

C. Device Compliance Dashboard

  1. Access via Intune admin centerReportsDevice compliance
  2. Monitor:
    • Compliance status by policy
    • Non-compliant device trends
    • Error patterns in compliance evaluation

Key Metrics to Track

  • Sign-in success rate: Should remain above 90%
  • Device compliance rate: Target 95%+ before full enforcement
  • Help desk tickets: Monitor for unusual spikes
  • User productivity impact: Track application access patterns

4. Rollback Procedures

Immediate Rollback Options

Option 1: Disable the Policy

  1. Navigate to the Conditional Access policy
  2. Change Enable policy from “On” to “Off”
  3. Takes effect within minutes for new sign-ins

Option 2: Switch to Report-Only Mode

  1. Edit the policy
  2. Change Enable policy to “Report-only”
  3. Maintains visibility while removing enforcement

Option 3: Exclude Affected Users/Groups

  1. Edit policy → AssignmentsUsers
  2. Under Exclude, add affected users or groups
  3. Use sparingly and temporarily

Grace Period Configuration

Configure grace periods in Intune compliance policies:

  1. Navigate to Intune admin centerDevicesCompliance policies
  2. Edit policy → Actions for noncompliance
  3. Set grace period (recommended: 3-7 days for initial rollout)
  4. Users maintain access during grace period while fixing compliance issues

Recovery from Deleted Policies

  • Deleted policies can be recovered within 30 days
  • Access soft-deleted policies through Microsoft Entra admin center
  • Restore maintains original configuration and assignments

5. Best Practices and Recommendations

Communication Strategy

  1. Pre-deployment: 2 weeks advance notice with requirements
  2. During pilot: Weekly updates to pilot users
  3. Rollout phases: 48-hour notice before including new groups
  4. Post-deployment: Success confirmation and support resources

Testing Checklist

  • ✓ Test with multiple device platforms (Windows, iOS, Android)
  • ✓ Verify enrollment process for new devices
  • ✓ Confirm excluded accounts remain accessible
  • ✓ Test rollback procedures in development environment
  • ✓ Validate help desk escalation procedures

Common Pitfalls to Avoid

  1. Not excluding emergency accounts – Can result in complete lockout
  2. Skipping report-only mode – Misses opportunity to identify issues
  3. Moving too quickly between phases – Insufficient time to identify problems
  4. Inadequate user communication – Leads to confusion and resistance
  5. Not monitoring device check-in intervals – Compliance updates may be delayed

PowerShell Monitoring Example


# Connect to Microsoft Graph
Connect-MgGraph -Scopes "Policy.Read.All"

# Get all Conditional Access policies
$policies = Get-MgIdentityConditionalAccessPolicy

# Filter for device compliance policies
$compliancePolicies = $policies | Where-Object { 
    $_.GrantControls.BuiltInControls -contains "compliantDevice" 
}

# Display policy status
$compliancePolicies | Format-Table DisplayName, State, CreatedDateTime

Implementing Risk-Based Conditional Access Policies for Small Business

Risk-based Conditional Access policies provide adaptive security that automatically adjusts authentication requirements based on the risk level of sign-ins and user behavior, helping you maintain an optimal balance between security and productivity.

Prerequisites and Licensing

  • Microsoft Entra ID P2 license required for risk-based policies (includes Identity Protection)
  • Microsoft 365 Business Premium includes Conditional Access features for small businesses
  • Users must be registered for Multi-Factor Authentication (MFA) before policy enforcement
  • Configure trusted network locations to reduce false positives

Step-by-Step Implementation Guide

Phase 1: Foundation Setup (Week 1)

  1. Create Emergency Access Accounts
    • Set up at least two break-glass accounts excluded from all policies
    • These prevent complete lockout if policies are misconfigured
  2. Start with Report-Only Mode
    • Deploy all new policies in report-only mode first
    • Monitor for at least 7-14 days to understand impact
    • Review sign-in logs to identify potential issues

Phase 2: Sign-in Risk Policy Configuration

  1. Navigate to Microsoft Entra admin center > Conditional Access
  2. Create new policy: “Require MFA for risky sign-ins”
  3. Configure settings:
    • Users: Include all users, exclude emergency accounts
    • Cloud apps: All cloud apps
    • Conditions > Sign-in risk: Select Medium and High
    • Grant: Require multi-factor authentication
    • Session: Sign-in frequency – Every time
    • Enable policy: Report-only (initially)

Phase 3: User Risk Policy Configuration

  1. Create new policy: “Require password change for high-risk users”
  2. Configure settings:
    • Users: Include all users, exclude emergency accounts
    • Cloud apps: All cloud apps
    • Conditions > User risk: Select High
    • Grant: Require password change + Require MFA
    • Enable policy: Report-only (initially)

Microsoft’s Recommended Risk Levels for Small Business

  • Sign-in Risk: Require MFA for Medium and High risk levels
    • Provides security without excessive user friction
    • Allows self-remediation through MFA completion
  • User Risk: Require secure password change for High risk only
    • Prevents account lockouts from overly aggressive policies
    • Users can self-remediate compromised credentials

Balancing Security and Productivity

Enable Self-Remediation

  • Sign-in risks: Users complete MFA to prove identity and continue working
  • User risks: Users perform secure password change without admin intervention
  • Reduces helpdesk tickets and minimizes productivity disruption

Progressive Deployment Strategy

  1. Pilot Group (Week 1-2)
    • Start with IT staff and power users
    • Monitor and gather feedback
    • Adjust risk thresholds if needed
  2. Phased Rollout (Week 3-4)
    • Expand to departments gradually
    • Provide user communication and training
    • Document self-remediation procedures
  3. Full Deployment (Week 5+)
    • Switch policies from Report-only to On
    • Monitor sign-in logs for blocked legitimate users
    • Fine-tune based on real-world usage

PowerShell Implementation Example

Import-Module Microsoft.Graph.Identity.SignIns

# Create Sign-in Risk Policy
$signInRiskPolicy = @{
    displayName = "Require MFA for risky sign-ins"
    state = "enabledForReportingButNotEnforced"
    conditions = @{
        signInRiskLevels = @("high", "medium")
        applications = @{
            includeApplications = @("All")
        }
        users = @{
            includeUsers = @("All")
            excludeGroups = @("emergency-access-group-id")
        }
    }
    grantControls = @{
        operator = "OR"
        builtInControls = @("mfa")
    }
    sessionControls = @{
        signInFrequency = @{
            isEnabled = $true
            type = "everyTime"
        }
    }
}

New-MgIdentityConditionalAccessPolicy -BodyParameter $signInRiskPolicy

Key Monitoring and Success Metrics

  • Sign-in Success Rate: Should remain above 95% for legitimate users
  • MFA Prompt Frequency: Monitor for excessive prompting that impacts productivity
  • Risk Detection Accuracy: Review false positive rates weekly
  • Self-Remediation Rate: Track percentage of users successfully self-remediating
  • Helpdesk Tickets: Should decrease after initial deployment

Best Practices for Small Business

  1. Start Conservative: Begin with High risk only, then add Medium risk after validation
  2. Communicate Clearly: Provide user guides explaining why MFA prompts occur
  3. Enable Modern Authentication: Block legacy authentication to prevent policy bypass
  4. Regular Reviews: Analyze risk detection patterns monthly and adjust as needed
  5. Document Exceptions: Maintain clear records of any policy exclusions
  6. Test Rollback Procedures: Know how to quickly disable policies if issues arise

Azure AD Identity Protection basics

Azure AD Identity protection is available with Azure AD P2 and provides risk detection and policy enforcement for sign ins and users. It can also be incorporated with Conditional Access policies to provide even more flexibility. This video shows you the basics of Azure AD Identity Protection as well as showing you an example of a login process that generates creates risk.

You can find the video here – https://www.youtube.com/watch?v=8AQQrSCrLMI

and more information here – What is Identity Protection?

Restricting user file downloads in SharePoint Online

https://www.youtube.com/watch?v=9NIcw5jghyA

There are situations with SharePoint Online where businesses wish to restrict users from downloading files. Unfortunately, this can’t be done at a document library level but I can be done at a user level provided you have licenses for Conditional Access.

Conditional Access is a features of Azure AD P1 and is included in SKUs like Microsoft 365 Business Premium. The above video takes you through the steps of configuring an appropriate Conditional Access policy in your environments to prevent downloads. The policy can be targeted at specific users and expanded to include other Microsoft 365 cloud services if desired.

PowerShell with Azure Conditional Access

Recently, I did a video demonstrating how PowerShell can be used to automate Endpoint Management:

PowerShell with Endpoint Manager

I’ve now also created a video demonstrating how to automate Azure Conditional Access using PowerShell. As before, I am only making these scripts available via the CIAOPS Paton program.

In this video you’ll see me automatically backup up both Conditional Access locations and policies, then apply best practices locations and policies, finally restore the original policies, all using scripting.

Again, these scripts are not free and part of the CIAOPS Paton program. You’ll find my free stuff at https://www.github.com/directorcia.

Conditional Access with Microsoft 365

One of the easy ways to protect your environment is to implement Conditional Access which is included with all Microsoft 365 plans. Otherwise, you can add Azure AD P1 to your environment to get this functionality.

This video will take you through the basics of setting up a Conditional Access including how to block access based on location. You’ll see how to create a Named Location, a Conditional Access policy and what it looks like when it is actually applied to a user.

A direct link for the video can be found at:

https://www.youtube.com/watch?v=TcYaRaNqYe4