During several of our incident response engagements, the various risky reports part of Azure Identity Protection proved valuable in identifying compromised users. However, we observed that these alerts frequently go unnoticed by our clients. To increase awareness about the Risky reports and their usefulness during incident response we decided to write a blog about them. The blog covers the following topics:
- Identity protection background
- Using information from Identity Protection for incident response
- Acquiring data from Identity Protection
Understanding Identity Protection
Microsoft Entra ID Protection helps organizations detect, investigate, and remediate identity-based risks. It uses advanced machine learning to identify sign-in risks and unusual user behavior to block, challenge, limit, or allow access.
The primary focus is on identifying suspicious behaviors that could indicate compromised identities or unauthorized access attempts. By providing detailed reports, Azure Identity Protection helps organizations proactively address security issues before they escalate.
The Four Risky Reports
Azure Identity Protection offers four main types of risky reports:
- Risky Users
- Risky Workload Identities
- Risky Sign-ins
- Risk Detections
You can access the Risky Reports by navigating to:
- Open your web browser and go to the Azure Portal.
- In the Azure Portal, search for Identity Protection.
- Under the Report section you can View the different Risky Reports.
An alternative approach is configuring the Diagnostics settings in Entra ID to forward the Risky sign-ins for both users and workload identities to Log Analytics/Sentinel.
Analysis
Now that we know where to find this information and the types of reports available, let's dive into each report type and understand their importance in incident response.
Risky Users
The Risky Users report lists all users whose accounts are currently or were considered at risk of compromise. This could be due to various reasons such as leaked credentials, unfamiliar sign-ins from new locations, or unusual activity patterns. A Risky user is reported when either or both of the following are true:
- The user has one or more Risky sign-ins.
- One or more risk detections are reported.
As shown in the screenshot below, a risky user named 15667323@bonacu.onmicrosoft.com was identified. In the header row, you have several immediate containment action options available.
- Reset password: This resets the password for the user and need to change at next login. Note, this doesn't kill existing sessions for a compromised user.
- Confirm user compromised: This setting moves the user to 'High' risk, however you still need to then have Conditional Access Policies that do something with a 'High' risk user otherwise nothing really happens. Again this does not magically solve an incident.
- Confirm user safe: This removes the risk associated with the user, and also provides feedback to Microsoft. Potentially dangerous as it allows an attacker to sort of prevent future alerts from triggering.
- Dismiss user risk: This just ignored this specific risk event, but doesn't affect Microsoft's future detections.
- Block user: Blocks the user in Entra ID.
Unfortunately, the details provided by Microsoft can often be limited. In this example, only one risky sign-in was detected, indicating unfamiliar sign-in properties. However, it does not specify which properties were identified as unusual. Due to the limited number of detections or sign-ins, this event was assigned a low-risk level.
To investigate this alert you can use the time information then click on the three dots top right and go to Sign-Ins:
During a recent investigation, we encountered a user with a high-risk level. Upon reviewing the user's details, we discovered multiple events, including an instance where an administrator confirmed the user as compromised. However, because no Conditional Access policies were configured, this action did not trigger any automated responses/remediation in the backend.
The Risk history tab also shows all the events that led to a user risk change in the last 90 days. This list includes risk detections that increased the user’s risk and admin remediation actions that lowered the user’s risk.
For your investigation you can use the sign-in events and specifically the following fields to identify any anomalies:
- Application, which application was used to sign-in, helps you find malicious application/consent grant attacks.
- Device, use this field to identify sign-ins using a non standard device for the given user.
- Geolocation, can be useful to find outliers e.g. logging in from a different country.
- IP Address, look out for logins via VPN IP-addresses and/or VPS providers.
- User-Agent, sometimes helpful if a user is always logging in with a certain device easy to find outliers.
The above analysis techniques enabled us to identify several Indicators of Compromise (IOCs), which we subsequently used to pivot and cross-reference with other evidence we had collected.
Risky Workload Identities
The Risky Workload Identities report focuses on non-human accounts, such as service principals and managed identities used by applications and automation tools. These identities are often targets due to their extensive access permissions. Additionally, most of the security solutions don't necessarily look at non-human identities or have a hard time finding suspicious ones.
Microsoft detects risks on workload identities through various mechanisms, focusing on sign-in behaviors and offline indicators of compromise. The following detection rules are in place:
- Microsoft Entra Threat Intelligence.
- Suspicious Sign-ins.
- Admin confirmed service principal compromised.
- Leaked Credentials.
- Malicious application.
- Suspicious application.
- Anomalous service principal activity.
We rarely encounter Risky Workload Identities during our investigations, so we are interested in hearing about your experiences with them. Do you come across these alerts in your investigations, and if so, have you found them to be useful?
Risky Sign-ins
The Risky Sign-ins report identifies individual sign-in attempts that contain suspicious behavior. This report helps pinpoint specific instances of potentially unauthorized access. The risky sign-in report contains events for up to the past 30 days. Based on this report you can find the following information that might be relevant during an incident response investigation:
- Which sign-ins are classified as at risk, confirmed compromised, confirmed safe, dismissed, or remediated.
- Real-time and aggregate risk levels.
- Detection types.
- Location information.
- Application information.
- Device information.
As shown in the screenshot below, our environment has several users flagged as at risk due to suspicious sign-in attempts.
Clicking on a detection reveals a bunch of additional useful information.
The basic information page offers some useful details, including the sign-in type, user, application, and IP address. As shown in the screenshot, you can view additional information about the device, risk level, multi factor authentication status, and conditional access policies. Each of these categories contains valuable insights that help determine whether the activity is malicious or legitimate. By analyzing event details such as the IP address and date, you can correlate this information with other log sources to build a comprehensive understanding of the incident.
Risk Detections
The Risk Detections report consolidates all detected risks across users and sign-ins, offering a view of potential security issues within the organization. While the Risky Sign-ins report provides only 30 days of history, the Risk Detections report extends this to 90 days. It includes relevant information such as:
- Detailed information about each risk detection.
- The location of sign-in attempts.
- Other risks triggered simultaneously.
This extended data retention and detailed information help in analyzing and responding to potential threats.
As shown in the screenshot, this risky detection was triggered due to unfamiliar sign-in properties for the given user. This overview enables us to identify the reason for the detection, providing crucial information to analyze the logs further and determine whether the activity is malicious or legitimate.
Incident Simulation
Another observation is that triggering identity protection alerts isn't working properly. Microsoft has put up a helpful site here describes how to generate detections. However, it isn't working we've performed plenty of logins via TOR without triggering anything. Additionally, we've published our application credentials on GitHub and no detections were triggered for workload protection.
Anyway we might be doing something wrong, but if someone has any insights on this please let us know.
Extraction of Identity protection alerts
As with (almost) every log source in Microsoft, the Microsoft Extractor can extract the relevant logs. If you prefer not to use the Microsoft Extractor, you can utilize the Microsoft Graph PowerShell module to retrieve the data yourself. Try it yourself:
- Open PowerShell
- Run Install-Module -Name Microsoft-Extractor-Suite
- Connect with Connect-Graph
At this moment in time you can use the Extractor-Suite to acquire:
- Risky users
- Get-RiskyUsers
- Risky detections
- Get-RiskyDetections
- Risky sign-ins
- Part of the Entra Sign-in logs
- Risky workload identities
- Work in progress
Conclusion
From our perspective Identity Protection and the associated Risky reports, serve as a potential indicator for a compromise. It is a quick and easy starting point to check if Microsoft has flagged any risky sign-ins, workloads, or users. This information provides a starting point for your investigation, allowing you to identify relevant users and IP addresses without beginning from scratch. As a point of improvement we would like to see just two categories of risk, one for users and one for non-human users instead of splitting out users at risk and risky sign-ins. We understand that not each user which is at risk is due to a sign-in event, but that can be solved in the event that is generated. Additionally, it would be nice if for the sign-ins that are classified as risky that this information would be present in the portal. At the moment when you export the Sign-In logs we do have the riskState field, but it's not shown in the Entra Portal.
A small tip for incident response whenever you're performing an investigation hop over to Identity Protection and you might find a lead for your investigation.
Want to know more about incident response in Microsoft cloud environments take it at your own pace via our OnDemand offering.