27 June 2023

Introduction

Long time has passed since Microsoft implemented the first Multi-Factor Authentication (MFA) approach in Azure Active Directory with the Per-user MFA functionality [1]. However, this simple on/off mechanism has been replaced over time by the Conditional Access Policy (CAP) feature, which was released on July 2016.

A conditional access policy is a set of conditions which, if matched, enforces its access controls to the assigned users if they try to access to the scoped applications. Access controls can block access directly, or grant access if some checks are met, such as the user completing the MFA validation or the accessing device being compliant. Users can be assigned individually, through security groups, or through roles.

Conditions within a conditional access policy are AND-wise. This means that the policy will apply only in those cases where all the conditions specified match. Some of these conditions have fixed values, while others such as device filters are more customizable. Also, conditions such as Locations and Device platforms have two lists: one for inclusions and other for exclusions.

However, conditions are not the only piece in determining whether a conditional access policy applies to a given user or not. Scoped applications could be set from the All cloud applications setting, which covers any access to the AAD controlled applications including the Microsoft 365 ecosystem, to single applications. There is even a choice of scoping by user actions and authentication contexts instead of applications.

Finally, it is worth noting how CAPs interact among them. Basically, if a sign-in event from a given user is covered by more than one CAP, all of them apply. This means that all the grant access controls among the applying policies will be requested (MFA, device compliance…). Policies configured with the deny access control have priority and the access will be just denied if at least one of these applies [2].

All these variables provide great granularity to the CAP feature, but this flexibility comes with a cost: there can be so many factors to consider when evaluating if a user will be asked to comply with the access controls, that there is no option to check if users have MFA enabled or not in an easy way, like the per-user MFA was. For example, one could find situations in which the same user is asked to perform multi-factor authentication if they try to sign-in to Exchange using a web browser, but not with the Desktop client. Such situations are known as ‘gaps’, which may grow exponentially as a given tenant contains more users, groups, and conditional access policies.

Tools to assist in determining the MFA status

Currently, there are a few tools that may help in the task of identifying gaps coming from conditional access policies, each following a different approach:

Azure Portal: sounds obvious, but Azure Portal provides a good insight of the overall MFA status. The Overview section in the Conditional Access blade offers security alerts that includes the percentage of sign-ins out of scope of CAPs and the percentage of sign-ins lacking MFA. Also, the sign-in logs in the same blade is quite useful to filter sign-ins by authentication requirements, and each entry can be drilled-in to analyse which CAPs were applied and which ones were not. However, it is limited to data generated by sign-ins in the last month period, meaning that it could miss MFA conditions if users did not login in that period, or if they already have a session opened that does not require re-authentication.
Conditional Access Gap Analyzer Workbook [3]: a tool aimed for IT Administrators that works similar to the Azure Portal. The main difference is that sign-in logs are stored in a Workspace Analytics resource, solving the time-limited problem that is present in the portal. Results are still non-deterministic since it relies on sign-in events. From the auditor’s position, this tool cannot be used since it requires a resource to be created in advance, unless agreed beforehand.
Azure AD Assessment [4] and Monkey365 [5]: although these tools are totally different, they follow the same approach regarding CAP analysis. In this case, the tools access to the tenant CAPs directly and determine if some best security practices have been applied, such as the existence of a CAP for every user and another for Global Administrators, but they do not include gap analysis.
CAOptics [6]: this tool was designed for CAP gap analysis specifically using a smart approach: it retrieves the tenant CAPs, then it generates permutations to represent each set of conditions for each CAP and affected user, group or role, indicating if the permutation has a termination (it is covered by one or more CAPs) or not. These permutations are then merged and gaps are exposed in form of unterminated permutations.
Azure-AD-Password-Checker [7]: a script that uses a genuine approach to raise potential MFA gaps by getting each user creation date and password change date, then comparing those in search of anomalies that represent a lack of MFA configuration by the user.

Per-user conditional MFA tool

From an auditor’s point of view, it would be really interesting to be capable of getting a deterministic, accurate report of the MFA status for each user in a target tenant. From the tools explained above, the only ones that can be catalogued as deterministic are Azure AD assessment, Monkey365 and CAOptics. The latter is the only one in this category that also focus on raising gaps, but its output is not per-user oriented.

This situational information is useful not only for customers from a defensive perspective, but also for attackers, even more now that Microsoft just enforced number matching and any attempt to access to an account with guessed credentials and MFA enabled will be quite unsuccessful.

Considering all of these, a decision was made to create a new plugin for the ROADrecon tool [8] that would receive a CAOptics report as input to generate a per-user MFA status report. There are multiple tools available for the public that already perform Azure AD security analysis, but ROADrecon was pretty good for our purpose since it implemented a plugin system and a really useful data model fed by a local database that plugin developers do not need to handle.

Transforming the input data into a per-user MFA list was not a simple task. For this reason, the plugin was designed to execute three main phases: the first one ingests the output data generated from CAOptics, the second one applies post-processing to enhance the per-user MFA status report and the last one is the output generation.

Input processing phase

The initial phase could be divided in two major steps. The first one is the input parsing and row mapping with its permutation, also called “lineage” in CAOptics. Since a per-user approach is going to be followed and the input report contains object IDs not only for users, but also groups and roles, these must be “unrolled” so that the permutation list only contains user IDs.

Unrolling groups and roles

Unrolling groups and roles may sound trivial, but Azure Active Directory does not specify a limit in the maximum depth for nesting. To make it worse, a child group can include any of its parent groups as member, generating loops in the tree that could derive in infinite lookup tasks. CAOptics already considered this situation and implemented the most efficient approach, which consists in establishing a reasonable limit of one level of depth.

Since we wanted to reach more accurate results even for “infernal” scenarios, and also considering that CAOptics was designed to not resolve role memberships into users, we decided to implement a lookup functionality to get a result that would fit better in the per-user approach.

Given an object ID, let’s call it the root node, the lookup algorithm would first check which kind of object it is dealing with. If it is a user, no action is required. If it is a group or a role, then the node is expanded, meaning that their members are retrieved as child nodes, and for each of these nodes, the same procedure is applied recursively. To prevent infinite loops, nodes that have been already expanded are cached into an expanded nodes list which is going to be checked by the recursive function before calling to itself again.

Once a root node and his children has been expanded, the final relationship is root node -> list of all its children user object IDs, which is cached into a resolved nodes cache for efficiency. This lookup procedure comes from Graph Theory and it is known as Depth-First Search (DFS).

For a given permutation being resolved, if the lookup procedure returns a list of multiple object IDs, the permutation is replaced by multiple copies of the same permutation, each containing a single object ID belonging to one of the returned users.

Determining the MFA approach

Getting the MFA status for every user also depends on the policy design, which can follow the include based or the exclude based approach. CAOptics works for the latter [9] and this must be considered when a tenant is found that follows the include based approach.

This is where the second major step in the input processing phase comes into play. Once all permutations are parsed, the plugin determines if there is a “main” MFA policy or not by examining the users:All lineage and terminations. We call the main policy to the CAP that is scoped to all users and all cloud applications. If there is such policy in place, all users are initially marked as MFA Enabled and then permutations without terminations are used to modify this status to Conditional or Disabled. When no main policy is detected, all users part from the MFA Disabled status and then their status is modified to Conditional or Enabled when examining their particular permutations.

The data model in ROADrecon already implements a strongAuthenticationDetail field for storing information about MFA, mostly focused on the legacy per-user MFA feature. The plugin extends this field with new attributes such as CapMfaStatus and CapMfaList to store the new information without overwriting the original data.

Post-processing phase

Up to this point, the plugin has a preview of the conditional per-user MFA status based in CAOptics results. However, since both tools differ from the output approach, a bit more of fine tuning is needed to make the report more accurate.

In first place, policies are processed individually to check if they have any influence or not. Those that are configured as Report only or Off are skipped. The same applies to policies that have no grant/deny controls or have an undefined scope. If a policy applies, then it is associated with every scoped user.

From that point, those conditions that have not been included in the MFA checking process are processed: authentication context scopes, devices, user risks, sign-in risks and locations conditions. If any or multiple of these configurations are detected, every user assigned to that policy will be updated to MFA Conditional status and the extra condition will be added to the report. For those users that were already marked as MFA Enabled, this process is omitted since the most restrictive policy wins.

Lastly, additional notes are added for those users that are affected by a blocking policy. To be more precise, the policy name will be appended to the blocking CAPs list of those users, but the MFA status is not updated here. This is because CAOptics already treated the blocking policies as MFA grant policies. While this may not be the most accurate approach for the plugin output, it will still reflect the MFA gaps with that extra information, which is enough for the purpose of this plugin.

Usage and output

Some prerequirements must be met before using the plugin. The first step requires getting the report from CAOptics to be used as input for the import plugin. It is important to use the --allTerminations flag, otherwise the report will not be accepted. Example of CAOptics execution line:

node ./ca/main.js --mapping --clearTokenCache --clearMappingCache --allTerminations

The report will be generated in two formats, CSV and MD. The CSV version will be used as input for the plugin.

Then we can move to ROADrecon and issue the authentication command. Currently, the plugin is only available in the plugin developer’s repository (https://github.com/acap4z/ROADtools), but a pull request to the main repository is going to be issued. It is important to note that the user must have the policy.read.all privilege assigned through a role such as Global Reader:

python .\roadrecon\roadtools\roadrecon\main.py auth --device-code

Once the tool has the authentication token, it can perform the tenant enumeration with the following command:

python .\roadrecon\roadtools\roadrecon\main.py gather --mfa

Finally, the CAOptics import plugin can be launched. By default, it will look for the CSV report in your current directory, but the path can be specified with the --input_file flag:

python .\roadrecon\roadtools\roadrecon\main.py plugin caopticsimport --input_file caoptics_report.csv

The final report will be written in a separate CSV file called output_report.csv by default, although this can be changed with the --output_file flag. There is also an option of getting a console output by specifying the --print flag, which displays a color code depending on the MFA status, but keeps additional info out such as conditions and CAP lists.

The CSV version contains more details than the printable version in the following columns:

User Principal Name: list of all users registered in the tenant, including guests.
MFA Status: the status obtained from the processing and post-processing phases for each user. Values can be:
- Enabled: one or multiple CAPs affecting the user covers every sign-in case with MFA.
- Conditional: the user is affected by one or more CAPs, but there are cases that are not covered by MFA.
- Disabled: the user is not affected by any CAP that manages grant controls.
MFA Bypass Conditions: when a user is marked with MFA Status conditional, the gaps will be listed in this column. Note that the ones coming from CAOptics will be more precise than those detected by post-processing tasks.
Blocking CAPs: name of the CAPs with grant controls set to Block that affect the user.
Affected by CAPs: list of all CAP names that affect the user.

It is important to remark that the MFA Status reported by the plugin does not consider the legacy per-user MFA status. Thus, it is possible to find tenants in which some users are reported with MFA Status Disabled, but their MFA has been enforced in the per-user MFA configuration. Microsoft recommends switching to conditional access to prevent such confusion in the MFA management [10].

The tool is currently available at the plugin developer’s repository: https://github.com/acap4z/ROADtools

Acknowledgements

Big thanks to those workmates that helped me with this research process. Special thanks to Simone Salucci, Daniel López and Manuel León for reviewing this post and suggesting me some meaningful improvements.