コンテンツにスキップ

Amazon Detective

Introduction

Welcome to the Amazon Detective Hub Best Practices Guide. The purpose of this guide is to provide prescriptive guidance for leveraging Amazon Detective for investigating security issues associated with your AWS resources. Publishing this guidance via GitHub will allow for quick iterations to enable timely recommendations that include service enhancements, as well as the feedback from the user community. This guide is designed to provide value whether you are deploying Detective for the first time in a single account, or looking for ways to optimize Detective in an existing multi-account deployment.

How to use this guide

This guide is geared towards security practitioners who are responsible for monitoring and remediation of threats and malicious activity within AWS accounts (and resources). The best practices are organized into categories for easier consumption. Each category includes a set of corresponding best practices that begin with a brief overview, followed by detailed steps for implementing the guidance. The topics do not need to be read in a particular order:

What is Amazon Detective?

Amazon Detective makes it easy to analyze, investigate, and quickly identify the root cause of security findings or suspicious activities. Detective automatically collects log data from your AWS resources and uses machine learning, statistical analysis, and graph theory to help you visualize and conduct faster and more efficient security investigations.

Who is Amazon Detective for?

Amazon Detective works well for customers who want to take advantage of a managed security service that helps connect the dots for investigating security issues in AWS. Many customers may be a bigger organization with many tools and want to enable analysts to quickly triage an incident. Other customers may have a leaner IT Security staff and need an effective tool to help their teams efficiently investigate security issues in AWS. Detective is for security analysts, security operations engineers, and incident response teams. Larger customers can benefit from using Amazon Detective if they have experienced tool sprawl and are looking to consolidate their SIEM tooling. Customers have particularly found value with Amazon Detective for telemetry and visibility into Amazon Elastic Kubernetes Service (EKS) clusters also, due to the tight integration of EKS audit logs and VPC flow logs.

What are the benefits of enabling Amazon Detective?

Detective aids incident responders and security engineers in determining what resources are associated with a given security event. It uses data feeds from GuardDuty findings, VPC Flow Logs, CloudTrail, EKS audit logs, and AWS Security Hub CSPM findings, to build a behavior graph to help customers understand resource associations, abnormal activity like newly observed geo locations, understanding when roles were assumed by other roles, along with many other use cases in your environment. Many customers may not have the log and telemetry collection correlation tooling built, or the knowledge or resources, and this makes Amazon Detective a perfect fit.

Many customers that have the tools, knowledge, and people already in place, are able to successfully navigate security events and discover related logs within an acceptable response time. Typically this requires third party tooling to be installed, configured, operated, and monitored to collect VPC Flow logs, CloudTrail logs, EKS Audit logs, and shipped to a log aggregation tool. Consider the cost tradeoff, and time saved, by using Amazon Detective where visibility into these data feeds is built-in vs aggregated and stored separately. Most customers have found Detective effective in shortening investigation times because they do not have to write queries into a log aggregation tool and instead rely on Detective provided insights.

Many customers observe that using Detective and GuardDuty together provides the ability to investigate natively in AWS after an event. GuardDuty tells you about the threat, but Detective tells you the story around that threat. Often we hear from customers that the amount of security findings from any given service overwhelm their teams and leave them without a clear priority. Detective enables teams to focus on true security events.

Detective is well suited Amazon EKS workloads to assist in runtime investigations. Detective allows you to view inside EKS workloads to examine pod configuration, pod image, image source, VPC flows within an EKS cluster, and activity by Kubernetes user within a cluster. If you run Amazon EKS across many AWS accounts, Detective’s ability to ingest data from the backend and centralize in a centralized Security Account helps reduce time to root cause during a security event. Take a closer look at our Container Security Workshop for more in-depth walkthroughs on how to use Detective for container based workloads.

Where Detective fits

AWS has added investigation capabilities in several places, so the most common question we now get is not how to use Detective but when to reach for it. Use this section to decide where Detective belongs in your process before you deploy it.

Reach for Detective when the question is "what exactly happened." Detective's differentiator is depth over raw log data. It maintains up to a year of aggregated activity from CloudTrail, VPC Flow Logs, EKS audit logs, and security findings in a behavior graph you can traverse interactively. When you need to follow a specific IP to an instance, to its instance profile, to the role that created it, to the federated user behind that role, and then check whether that user's behavior was normal three months ago, that is Detective. No summary answers that question, because the next pivot depends on what you found in the last one.

Reach for Detective for EKS investigations. This is where the depth advantage is largest. Detective lets you examine pod configuration, pod image and image source, VPC flows inside a cluster, and activity by Kubernetes user, correlated across accounts. If you run EKS at scale, this visibility is difficult to reconstruct from raw logs under time pressure.

Start elsewhere when you need a fast first assessment. If you want a structured disposition on a specific GuardDuty finding, with a confidence rating, MITRE ATT&CK classification, and recommended actions, GuardDuty Investigation produces that in minutes and is the more efficient first step. If you are trying to understand which risks in your environment matter most, or how an attacker could chain misconfigurations and vulnerabilities together to reach something important, the exposure findings and attack path graph in AWS Security Hub are built for that question and Detective is not.

A reasonable operating model for teams running all three: triage and prioritize in Security Hub, get a fast first read on a specific threat finding with GuardDuty Investigation, and bring in Detective when the assessment raises a question that only the underlying activity data can answer. Detective is the depth tool at the end of that sequence rather than the entry point, and scoping it that way also keeps its cost proportionate to how often you actually need that depth.

Getting started

Enable GuardDuty

Enabling Amazon GuardDuty is a pre-requisite before enabling Amazon Detective. Follow the Amazon GuardDuty documentation for enabling GuardDuty and enrolling AWS Accounts in your Organization at scale.

If you run Amazon EKS in your environment, it is recommended to enable EKS Audit and Runtime Protection in GuardDuty. In the enablement recommendations below we will discuss consuming EKS audit logs in Detective.

It’s recommended to update the GuardDuty CloudWatch Notification Frequency from its default of 6 hours to 15 minutes. If you leave it at 6 hours, it could take updates to recurring findings up to 6 hours to be reflected in Detective. Regardless if you do or do not change this frequency, new findings are always sent to Detective within five minutes of the finding being generated. Shortening this timeframe does not incur any additional costs. For information about setting the notification frequency, see Monitoring GuardDuty Findings with Amazon CloudWatch Events in the Amazon GuardDuty User Guide.

Where to enable Detective

We recommend enabling Amazon Detective in the same Delegated Administrator AWS Account as the rest of your AWS Security Services (GuardDuty, Security Hub, Macie, etc), as outlined in the Security Reference Architecture (SRA) whitepaper. If you don’t have a dedicated Security AWS Account yet, this is the perfect opportunity to create one. If you run AWS Control Tower, you already have one which is often called the “Audit” or “Security” Account.

Note that Detective currently supports a maximum of 1200 AWS accounts in each behavior graph. If you’re interested in enabling Detective for more than 1200 accounts please reach out to your AWS account team. For other Detective quotas refer to the documentation on regions and quotas.

Region Considerations

Amazon Detective is a regional service. This means that to use Amazon Detective you will need to enable it in every region that you would like to have incident response capabilities with Detective.

A question that is often asked is “Should I use a security service in a region that my company is not actively using?” Detective’s costs are based on the amount of data ingested in a given account and region. If an AWS account or region has no activity, Detective will not incur a cost. It’s recommended to review the Usage section in Detective to determine if the estimated costs are acceptable. We recommend enabling Detective in every AWS account and region that you plan to make available in your AWS Organization. This ensures that if someone attempts to create resources in a region, you will have visibility if needed for any subsequent security issues.

You can use the AWS console, AWS CLI, or the Amazon Detective python scripts to enable Detective across accounts in an AWS Organization. Review the scripts and their contents here.

If your organization requires that access to AWS services stay on the AWS network, Detective is integrated with AWS PrivateLink and supports interface VPC endpoints. This matters for incident response specifically, because responders often work from a hardened environment with restricted internet egress, and requiring a public endpoint to reach your investigation tool is a poor constraint to discover mid-incident.

Implementation

Enablement

To create a Delegated Administrator for Amazon Detective to your desired Security Tooling Account, follow the enablement documentation. Then, using your AWS console, open your Security Tooling account and browse to the Detective service.

Detective is very easy to enable. There is nothing to install and the integration with other AWS services is seamless. In the Detective service, enable Detective by clicking on the orange Get Started button.

Detective getting started

Figure 1: Detective getting started page

Review the service linked role that Detective will create to understand the permissions needed and then click enable Amazon Detective.

Detective Enablment

Figure 2: Enable Detective

Enroll other AWS Accounts in your Organization

After enabling Detective you will need to enroll other AWS Accounts in the Organization. In the Detective console browse to Account Management. From the Account Management page you can enroll all accounts in your organization by selecting the “Enable all accounts“ button pictured below.

Enable all accounts

Figure 3: Enable all Detective accounts

After you have enabled for all accounts in your organization ensure that you have turned on the "auto-enable" button as pictured below. This give the Detective delegated administrator the ability to automatically enable Detective in all new accounts that are created in your organization without needing to perform any actions by your team.

Detective auto enable

Figure 4: Detective auto enable feature

Enable Optional Data Feeds

Browse to the General section.

If you have just enabled Detective, you will already be ingesting findings from AWS security services and EKS audit logs. If you were using Detective before these integrations were released you will need to enable them manually.

Update Source packages

Figure 5: Detective source packages

Enable Amazon Security Lake integration

Optionally, you can configure Detective to integrate with Amazon Security Lake to query and retrieve logs stored in Security Lake directly in the Detective console. With this integration, security analysts can start their security investigations based on summaries and visualizations in Detective. In the event they need to dive deeper and retrieve logs, Detective provides a pre-built query using Amazon Athena that is scoped to the time and entities under investigation. You can use this query to get a preview of CloudTrail or VPC flow logs that represent the potential security issue or download all the logs as CSV files.

SL Integration Figure 6: Security Lake integration enablement page

Operationalizing

Providing Access

Amazon Detective works by consuming VPC Flow Logs, Cloudtrail Logs, EKS Audit Logs, and AWS Security Hub CSPM findings and correlating resource IDs with events from GuardDuty into a behavior graph. This graph is what you can follow in Detective when performing an investigation to see which IP relates to which EC2 instance to which EC2 instance profile to which IAM role that created the instance profile, for example.

Detective flow

Figure 7: Detective workflow diagram

As a result, there may be information being fed into Detective in the form of IP addresses, package vulnerabilities, IAM roles and federated users. You will want to apply least privilege to your Incident Investigator IAM Role.

Most customers will opt to provide access to their security and incident response teams to the Security Tooling account where many AWS Security Services including Security Hub, Inspector, Macie, and GuardDuty delegated administrators have been assigned. It’s best practice not to provide any team full administrative privileges to the AWS Account, and security teams are no different. We recommend granting incident responders using Detective least privilege rights to do their job, and a great way to start with this is by attaching the AWS IAM Managed Policy AmazonDetectiveInvestigatorAccess to your incident responder’s IAM Role. You can view this policy, and others, on the Settings > General tab.

Detective IAM policies

Figure 8: Detective IAM policies

Investigating a Finding

There are four primary ways to start an investigation using Detective. We'll cover these methods in detail below.

  1. Using the Investigate with Detective option from GuardDuty
  2. Examining a Finding Group in Detective
  3. Threat hunting using Detective's Search and geography graphs
  4. Pivoting into Detective from a SIEM using an embedded URL

Detective supports GuardDuty attack sequence findings from Extended Threat Detection, which are the Critical findings GuardDuty generates when it correlates a multi-stage attack. Those are usually the highest-value findings to bring into Detective, because an attack sequence tells you a chain of events occurred and Detective is where you establish exactly which resources and identities were involved at each step.

Start your investigation from GuardDuty

Most customers start investigations after a threat has been detected through Amazon GuardDuty, using the Investigate with Detective option on the Finding. If you don't have any GuardDuty findings present, you can still use Detective to investigate activity or threat hunt, as we’ll explain in method #3 below. When you open a GuardDuty finding, a detail pane will appear on the right. Click the Investigate with Detective option.

GuardDuty finding

Figure 9: Investigation link in GuardDuty finding detail page

You’ll see a panel with all the related resources collected by Detective. Let’s start with the GuardDuty finding.

Detective investigation options

Figure 10: Detective investigation options from GuardDuty finding

From here we can see the VPC ID, the subnet ID, and other related information.

Detective entities

Figure 11: Entities related to GuardDuty Finding

Let’s browse into the EC2 instance and see the related information.

Detective EC2 Instance

Figure 12: Instance profile page in Detective

From here you will be able to determine when the EC2 was built, the AWS Account it resides in, the IAM Role or User that created the EC2 instance.

If we click into the Created By IAM Role, we can look at the Resource Interaction tab, identify the IAM identity that created this EC2 instance, when the first observance of the user was, the most recent occurrence, and the various times this user has assumed this role during the scope time.

Detective Role

Figure 13: Role profile page in Detective

For the sake of this walkthrough let’s assume you have noticed a strange occurence with the role assumptions of the user in question. With that in mind at this point you may want to execute your incident response plan, isolate and snapshot the EC2 instance, revoke credentials for past sessions on the IAM Role, and further investigate the federated user who created the instance. If your organization does not currently have incident response playbooks, there are publicly available samples to help you get started.

Start your investigation from a Finding Groups

Amazon Detective Finding Groups enable you to examine multiple activities as they relate to a single security event. If a threat actor is attempting to compromise your AWS environment, they typically perform a sequence of actions that lead to multiple security findings and unusual behaviors. These actions are often spread across time and entities. When security findings are investigated in isolation, it can lead to a misinterpretation of their significance and difficulty in finding the root cause. Amazon Detective addresses this problem by applying a graph analysis technique that infers relationships between findings and entities, and groups them together. We recommend treating Finding Groups as the starting point for investigating the involved entities and findings. While GuardDuty can show you the threat event, Detective can tell you the story of what else happened in the environment related to the event.

When we first navigate to a Finding Group (example image below), we can see that we are presented with a generative AI powered summary of the Finding Group. This gives us an understanding of the activity associated with the Finding Group before we even dive into the data.

Detective Gen AI Figure 14: Detective generative AI Finding Group summary

Detective will attempt to correlate threat events and resources to the MITRE ATT&CK framework. MITRE publishes a matrix of Tactics, Techniques and Procedures (TTPs) that malicious actors perform when attacking a system. These correlated TTPs are published by Detective into a Finding Group. This is a visual representation of the events, resources, and security findings that make up an event.

Detective Finding Group TTPs

Figure 15: TTPs associated with Detective Finding Group

The Finding Group is represented by this generated graph that provides a visualization of associated resources and how they are connected. In this example, there are many different IP addresses associated with a brute force GuardDuty finding, and some of these resources are associated with other GuardDuty findings, like this Domain Generation Algorithm (DGA) request. DGAs are used to periodically generate a large number of domain names that can be used as rendezvous points with their command and control (C&C) servers. During a GuardDuty finding investigation, it might not be clear that there are other resources included in the overall security issue. Detective Finding Groups help connect these dots.

Detective Finding Group graph

Figure 16: Detective Finding Group graph visualization

Use these Finding Groups to investigate other IPs, EC2s, or AWS accounts from where an attack originated or continued onto.

The visualization also offers a timeline layout in addition to the graph layout, with a play control that steps through events in sequence and filtering by severity. The graph view answers what is connected to what. The timeline answers what happened in what order, which is usually the harder question during an investigation and the one you need for an incident writeup. If you are trying to establish whether the privilege escalation preceded or followed the initial access, start in the timeline rather than the graph.

The data correlating events to resources is available in raw data if you are collecting these data sources individually, but security analysts need to know how to craft individual queries to crawl through logs and then graph to a visualization tool later. Detective short-circuits this investigative effort and query generation activity. While most customers don’t replace their traditional SIEM tools with Detective, the use of Detective with a SIEM is effective at reducing investigation times.

Compare Detective's effectiveness to collecting and aggregating your logs separately. If you are collecting data sources individually and storing in a log aggregation tool, your security analysts will need the expertise to craft queries to find the data they need, and graph to a visualization tool separately. Detective short-circuits this alert --> log --> query --> visualize process by providing you a visualization using built-in AWS data feeds. While most customers don’t replace their traditional SIEM tools with Detective, the use of Detective with a SIEM is effective at reducing investigation times.

Threat Hunting in Detective

Customers can use Detective if they have interesting information that they want to try to correlate to AWS resources or events. Commonly, Detective is used to investigate where your AWS resources are communicating with suspicious public IPs. Browse to Amazon Detective console, on the left navigation pane click the Search option. You can search for many kinds of resources including EC2 instances, IP addresses, Kubernetes Subjects, or Container Images.

Detective search options

Figure 17: Detective search options

Detective IP search

Figure 18: Searching for IP in Detective

Important Note: With regards to IP addresses, it’s important to remember which IPs are yours vs public IPs. Most customer IPv4 addresses are in the RFC1918 range (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16). If you are using public IPs in your IP schema for your VPCs, you may need to perform lookups occasionally to determine who owns the IP.

In addition to searching for information Detective Investigations lets you investigate IAM users and IAM roles using Indicators of Compromise (IoC). IoCs can help you determine if a resource is involved in a security incident. Detective investigations enables you to investigate attack tactics including impossible travel, flagged IP addresses, and Finding Groups. It performs initial security investigation steps and generates a report highlighting the risks identified by Detective to help you understand security events and respond.

To start an investigation, select Run Investigation. Choose your desired date/time and confirm.

Detective Start investigation Figure 19: Start investigation button on IAM role entity page

Detective confirm investigation Figure 20: Confirm investigation settings

Once the investigation has run, new information will be available including TTPs, threat intelligence flagged IP addresses, impossible travel, related Finding Groups, related findings, new geolocations, new user agents, and new ASOs. All information will be provided in the console and can be downloaded in a JSON format.

Detective investigation results Figure 21: Detective sample investigation results

Crafting Detective Embedded URLs

The fourth way to start an investigation is to construct a Detective URL and embed it wherever your analysts already work. Detective profile pages are addressable by URL, so if your SIEM or ticketing system holds the finding, you can attach a link that opens the corresponding Detective profile scoped to the right entity and time window. Analysts then move from alert to investigation in one click instead of switching consoles and re-finding the resource by hand.

This is worth setting up because it removes the most common reason Detective goes unused in organizations that own it. When the alert lives in a SIEM and Detective requires a separate console login and manual lookup, analysts skip it under time pressure. A link in the alert changes that default.

If Splunk is your SIEM, the Splunk Trumpet project sends data from AWS services to Splunk and can be configured to generate Detective URLs for GuardDuty findings, giving you this pattern without building it. For other tools, apply the same approach using your own alert enrichment.

Detective URLs

Figure 22: Crafting Detective embedded URLs

Cost Considerations

Amazon Detective pricing is thoroughly covered in the pricing page, including dimensions used for pricing and some pricing examples. Detective is billed on the volume of data ingested into your behavior graph, and it is billed separately from the AWS Security Hub essentials plan. That makes account scope your primary lever, since cost tracks the accounts and Regions you enroll rather than the number of investigations you run.

  1. Take advantage of the 30-day free trial to get an idea of how pricing works with Detective. After enabling Detective, come back in a few days and check the Usage tab on the left.
  2. In the Detective usage page you can view the estimated costs under the This account’s projected costs and All accounts’ projected costs. You can further understand which accounts are driving costs by using the Ingested volume by member account option. Because pricing follows ingestion, a small number of high-traffic accounts usually accounts for most of the bill, so start there rather than trimming broadly.
  3. Consider excluding accounts classified as sandbox or non-production. This is the most common scoping decision, and it is a real tradeoff rather than a free saving, since an excluded account is one you cannot investigate later. Weigh it against how likely that account is to be part of an incident.
  4. Look for duplicate ingestion. If you already pay a separate tool to collect VPC Flow Logs, CloudTrail, and EKS audit logs purely to support investigations, you are paying twice for the same telemetry. Either scale that collection down and lean on Detective for investigation, or scope Detective to the accounts where you do not already have that coverage. Deciding which of the two is cheaper depends on what else consumes those logs, so check with the teams that own the pipeline before changing either side.

Detective usage page

Figure 23: Detective usage page

Resources

Workshops

Videos

Blogs