Skip to content

Amazon GuardDuty

Introduction

Welcome to the Amazon GuardDuty Best Practices Guide. The purpose of this guide is to provide prescriptive guidance for leveraging Amazon GuardDuty for continuous monitoring of your AWS accounts and resources. Publishing this guidance via GitHub will allow for quick iterations to enable timely recommendations that include service enhancements, as well as, the feedback of the user community. This guide is designed to provide value whether you are deploying GuardDuty for the first time in a single account, or looking for ways to optimize GuardDuty in an existing multi-account deployment.

How to use this guide

This guide is geared towards security practitioners who are responsible for monitoring and remediation of threats and malicious activity within AWS accounts (and resources). The best practices are organized into three categories for easier consumption. Each category includes a set of corresponding best practices that begin with a brief overview, followed by detailed steps for implementing the guidance. The topics do not need to be read in a particular order:

What is Amazon GuardDuty?

GuardDuty is an intelligent threat detection service that continuously monitors your AWS accounts, Amazon Elastic Compute Cloud (Amazon EC2) instances, AWS Lambda functions, Amazon Elastic Kubernetes Service (Amazon EKS) clusters, Amazon Elastic Container Service (Amazon ECS), Amazon Aurora login activity, and data stored in Amazon Simple Storage Service (Amazon S3) for malicious activity. If potential malicious activity, such as anomalous behavior, credential exfiltration, or command and control infrastructure (C2) communication is detected, GuardDuty generates detailed security findings that can be used for security visibility and assisting in remediation. Additionally, using the Amazon GuardDuty Malware Protection feature helps to detect malicious files on Amazon Elastic Block Store (Amazon EBS) volumes attached to Amazon EC2 instance and container workloads.

GuardDuty monitors Foundational data sources such as AWS CloudTrail event logs, AWS CloudTrail management events, Amazon VPC Flow Logs, and DNS logs. Enablement of the Foundational data sources is not required. Amazon GuardDuty pulls independent streams of data directly from those services.

In addition to log data sources, GuardDuty can use additional data from other AWS services in your AWS environment to monitor and provide analysis for potential security threats. Those services include:

  • Amazon EKS – GuardDuty monitors EKS Audit Logs and operating system-level events via the GuardDuty security agent.
  • AWS Lambda – GuardDuty monitors network activity logs, including VPC Flow Logs for suspicious network traffic.
  • Amazon EC2 and Containers – GuardDuty monitors its findings for indicators of malware presence on attached Amazon Elastic Block Store (Amazon EBS) volumes, e.g. bitcoin mining activity.
  • Amazon Aurora – GuardDuty monitors and profiles relational database service login activity for potential threats.
  • Amazon S3 – GuardDuty monitors AWS CloudTrail S3 data events to identify potential threats in your Amazon S3 resources. AWS CloudTrail S3 management events are monitored by default after GuardDuty is enabled.

What are the benefits of enabling GuardDuty?

GuardDuty is designed to operate completely independently from your resources and have no performance or availability impact to your workloads. The service is fully managed with integrated threat intelligence, machine learning (ML) anomaly detection, and malware scanning. GuardDuty delivers detailed and actionable alerts that are designed to be integrated with existing event management and workflow systems. There are no upfront costs and you pay only for the events analyzed, with no infrastructure to manage or threat intelligence feed subscriptions required.

Deploying GuardDuty

Single account deployment

The minimum requirements for enabling GuardDuty as a standalone account or as a GuardDuty administrator with AWS Organizations are covered below.

The first step to using GuardDuty is to enable it in your account. Once enabled, GuardDuty will immediately begin to monitor for security threats in the current Region. For a standalone account:

  1. Open the GuardDuty console at: https://console.aws.amazon.com/guardduty/
  2. Choose Get Started. GuardDuty landing page Figure 1: GuardDuty landing page
  3. Choose Enable GuardDuty. Enable GuardDuty Figure2: GuardDuty enablement

Once these steps are complete GuardDuty will begin collecting log data and monitoring the environment for malicious or suspicious activity.

Multi-account deployment

As prerequisites for this process, you must be in the same organization as all the accounts you want to manage, and have access to the AWS Organizations management account in order to delegate an administrator for GuardDuty within your organization. Additional permissions may be required to delegate an administrator, for more information see Permissions required to designate a delegated administrator.

Enablement of a delegated administrator

  1. Open the AWS Organizations console at https://console.aws.amazon.com/organizations/, using the management account.
  2. Open the GuardDuty console at https://console.aws.amazon.com/guardduty/.

Is GuardDuty already enabled in your account?

  • If GuardDuty is not already enabled, you can select Get Started and then designate a GuardDuty delegated administrator on the Welcome to GuardDuty page.
  • If GuardDuty is enabled, you can designate a GuardDuty delegated administrator on the Settings page.

  • Enter the twelve-digit AWS account ID of the account that you want to designate as the GuardDuty delegated administrator for the organization and choose Delegate. GuardDuty Auto Enable off Figure 3: GuardDuty auto-enable off

Configuring auto-enable preferences for organization

  1. Open the GuardDuty console at https://console.aws.amazon.com/guardduty/.
  2. In the navigation pane, choose Accounts.
  3. The Accounts page provides configuration options to the GuardDuty administrator to Auto-enable GuardDuty and the optional protection plans on behalf of the member accounts that belong to the organization. You can find this option next to Add accounts by invitation.

This support is available to configure GuardDuty and all the supported optional protection plans in your AWS Region. You can select one of the following configuration options for GuardDuty on behalf of your member accounts:

  • Enable for all accounts – Select to enable the corresponding option for all the member accounts in an organization automatically. This includes new accounts that join the organization and those accounts that may have been suspended or removed from the organization.
  • Auto-enable for new accounts – Select to enable GuardDuty for new member accounts automatically when they join your organization.
  • Do not enable – Select to prevent enabling the corresponding option for any account in your organization. In this case, the GuardDuty administrator will manage each account individually. Manage Auto-enable preferences

Figure 4: GuardDuty auto-enable preferences

  1. Choose Save Changes

Add accounts as members to your organization

This procedure covers adding members accounts to a GuardDuty delegated administrator account through AWS Organizations. To learn more about associating members in GuardDuty, see Managing multiple accounts in Amazon GuardDuty AWS service integrations with GuardDuty.

  1. Log in to the delegated administrator account
  2. Open the GuardDuty console at https://console.aws.amazon.com/guardduty/.
  3. In the navigation panel, choose Accounts. The accounts table displays all of the accounts that are added either Via Organizations or By invitation.
  4. Select one or multiple account IDs that you want to add as members. Thses account IDs must have Type as Via Organizations.
  5. You can select the down arrow of the Status column to sort the accounts by the Not a member status and then choose each account that does not have GuardDuty enable in the current Region.
  6. Choose Confirm to add the accounts as members. This action also enables GuardDuty for all of the selected accounts. The Status for the accounts will change to Enabled.
  7. (Recommended) Repeat these steps in each AWS Region. This ensures that the delegated administrator can manage findings and other configurations for member accounts in all the Regions where you have GuardDuty enabled.

GuardDuty protection plans

After enabling GuardDuty in your account(s), choosing additional protection types is highly recommended. GuardDuty protection plans are additional features that add focused threat detection for Amazon EKS, Amazon S3, Amazon Aurora, Amazon EC2, Amazon ECS, and AWS Lambda. To learn more about the benefits of what each GuardDuty protection provides, refer to the protection section of the Amazon GuardDuty User Guide.

Enable protections

Similar to the Auto-enable feature to turn on GuardDuty in organization member accounts, protection plans can be Auto-enabled as well. To set Auto-enable, click the desired protection plan. Within the protection plan you can opt to 1. Enable for all accounts or 2. Configure accounts manually. To avoid gaps in protection coverage within your AWS organization, it is highly recommended to enable your desired protection across all accounts (active and new). Edit S3 Protection configuration Figure 5: GuardDuty S3 Protection configuration

Tip: Selecting Configure account manually allows the delegated administrator to manage protection enablement at the account level, per Region, with the option to Auto-enable for new member accounts to the organization

GuardDuty runtime monitoring

For GuardDuty runtime monitoring protections including GuardDuty runtime monitoring for EKS, EC2, and ECS you have the ability to further scope what resources you want coverage for and how you would like to deploy the runtime agent. We recommend allowing GuardDuty to deploy runtime monitoring which will deploy a VPC endpoint and agent for ECS and EKS using a sidecar or EKS managed add-on, respectively. This will ensure coverage across your current resources but also apply to new resources that are created in the future. Using the GuardDuty automation saves manual effort needed to address new resource coverage across your organization. However if you can choose to you can manage this configuration yourself. GuardDuty EC2 monitoring is currently in preview and only has the option to use systems manager or the RPM package manager to install the runtime agent.

If choose to allow GuardDuty to deploy the needed resources to cover both current and future VPC endpoints and agents needed to provide runtime monitoring in your environment you can further scope which resources the runtime monitoring applies to by using inclusion or exclusion tags. For example, if you have a handful of EKS clusters that you don't want to monitor, than it would be more efficient to use an exclusion tag rather than the opposite scenario where you only want to apply monitoring to a few resources where you would want to use inclusion tags. The GuardDuty documentation provides use case examples and specific tags needed to implement this functionality.

Operationalize GuardDuty findings

After GuardDuty has been enabled in your account(s), GuardDuty will begin monitoring the foundational data sources and analyzing features associated with optionally enabled resource protection types. A best practice recommendation after enabling GuardDuty is to leverage the 30-day trial (enabled per account) to understand the baseline of normal activity, derived by machine learning, for your accounts and resources. During the trial period GuardDuty will also use threat-based and rules-based intelligence to generate findings in near-real time.

Potential security issues are presented as findings in the GuardDuty console. All GuardDuty findings are assigned a severity level (low, medium, high) and corresponding value (1.0 – 8.9) based on potential risks. Higher value findings indicate a greater security risk. The severity level assigned to a finding is provided to help you determine a response to a potential security issue that is highlighted by a finding. GuardDuty Severity levels Figure 6: GuardDuty severity levels

It is recommended to familiarize your team with the GuardDuty Finding Types. This will give you insights into what findings you might see in your account, the details associated with these findings, and potential remediation actions. After you understand different GuardDuty findings we encourage customers to think through filtering, notifications, and potential automatic remediation which will cover in the next sections. It is also encouraged to begin writing incident response playbooks for responding to GuardDuty findings in your environment. This will likely combine multiple steps below and also details not covered that might be specific to your environment and your teams.

Filtering findings

If you are new to GuardDuty the easiest way to view/filter your findings is via the Summary tab. The Summary dashboard displays an aggregated view of the last 10,000 GuardDuty findings generated in the current AWS Region. You can choose to view the Last 2 days (default), Last 7 days or Last 30 days. This dashboard provides 6 widgets, 3 of which include filter capabilities to customize your view. GuardDuty Summary page Figure 7: GuardDuty summary page

As the delegated administrator becomes more familiar with their organizations’ GuardDuty findings, an advanced filtering technique may be employed using the Findings tab. The Findings tab exposes 80+ finding attributes that can match the criteria you specify or filter out any unmatched findings. A common filter technique is one that focuses on threats that have a high indication that 1. A resource has been compromised, for example looking at high severity findings 2. The type of finding indicates potential risks for unwanted billing charges, for example CryptoMining.

Surfacing Bitcoin mining is an example of one such finding filter that could be created. Bitcoin is a worldwide cryptocurrency and digital payment system that can be exchanged for other currencies, products, and services. Bitcoin is a reward for bitcoin-mining and is highly sought after by threat actors. To find out if any of your EC2 instances have been compromised for purposes of Bitcoin mining you can use this attribute and value pairing: Severity:High, Finding type:CryptoCurrency:EC2/BitcoinTool.B!DNS. Applying this filter will provide a view of any EC2 instances that are querying a domain name that is associated with Bitcoin or other cryptocurrency-related activity. GuardDuty Finding page Figure 8: GuardDuty finding page

Note: Frequently used filters can be saved to reduce future efforts. To save the specified attributes and their values (filter criteria) as a filter, select Save. Enter the filter name and description, and then choose Done.

Reducing potential noise

As accounts and workloads grow within an AWS Organization, it is possible there will be an increase in GuardDuty findings because of certain configurations. Some of the findings may be deemed low value or threats not intended to be acted upon. To make it easier to recognize security threats that are most impactful to your environment, enabling quick remediation actions, it is recommended to deploy suppression rules. A suppression rule is a set of criteria, consisting of a filter attribute paired with a value, used to filter findings by automatically archiving new findings that match the specified criteria.

After creating a suppression rule, new findings that match the criteria defined in the rule are automatically archived as long as the suppression rule is in place. Existing filters can be used to create suppression rules. Suppression rules can be configured to suppress entire finding types, or define more granular filter criteria to suppress only specific instances of a particular finding type. Suppression rules can be edited at any time. It is recommended to suppress with as much granualarity as possible. This will help with reducing findings for only certain circumstances without losing total coverage of a finding type.

Suppressed findings are not sent to AWS Security Hub, Amazon S3, Amazon Detective, or Amazon CloudWatch, reducing finding noise level if you consume GuardDuty findings via Security Hub, a third-party SIEM, or other alerting and ticketing applications.

GuardDuty continues to generate findings even when they match suppression rules, however, those findings are automatically marked as archived. An archived finding is stored in GuardDuty for 90-days and can be viewed at any time during that period. You can view suppressed findings in the GuardDuty console by selecting Archived from the findings table, or through the GuardDuty API using the ListFindings API with a findingCriteria criterion of service.archived equal to true.

A common use case for a suppression rule is filtering out a known resource that scans EC2 instances. This resource may result in a finding type Recon:EC2/Portscan which for certain resources in your environment may be intended. For this scenario it is recommended to suppress associated findings using a combination of resource tagging and Finding Type.

Automated notification for high priority findings

After creating suppression rule filters to allow for surfacing only the most impactful findings for your environment, the next recommended action is automating the notification of high priority findings. Amazon EventBridge is a serverless event bus that makes it easier to build event-driven applications at scale using events generated from AWS services. GuardDuty creates an event for Amazon EventBridge for newly generated findings. All findings are dynamic, meaning that, if GuardDuty detects new activity related to the same security issue it will update the original finding with the new information as a newly aggregated finding. Thus, creating a new event for EventBridge to potentially process. Delegated administrators can optionally change the frequency of publishing new findings to EventBridge in GuardDuty Settings under Findings export options. By default, EventBridge is updated every 6 hours but can be changed to 1 hour or 15 minutes intervals.

Every GuardDuty finding is assigned a finding ID. GuardDuty creates an EventBridge event for every finding with a unique finding ID. By using Amazon EventBridge with GuardDuty, you can automate tasks to help you respond to security issues related to GuardDuty findings. One such task is sending notifications to a preferred notification channel. Commonly used notification channels are email and Enterprise messaging applications that support incoming messages via Webhook.

Another common scenario is to send GuardDuty findings to a ticketing system or SIEM solution for tracking and event correlation. It is recommended to leverage Security Hub as a central aggregation point in AWS for security findings before sending these findings to a ticketing system or SIEM solution. You can also leverage EventBridge to send GuardDuty findings directly to a ticketing system or SIEM solution. Please refer to your solutions documentation for guidance on how to send information via EventBridge.

When leveraging email as a preferred notification channel, GuardDuty is integrated with Amazon Simple Notification Services (Amazon SNS) via EventBridge. Amazon Simple Notification Service (Amazon SNS) is a fully managed messaging service for both application-to-application (A2A) and application-to-person (A2P) communication. The notification workflow is as depicted in the image below. GuardDuty Notification workflow Figure 9: GuardDuty notification workflow

All GuardDuty findings not matching a suppression rule are automatically delivered to the account event bus. There are only two configurations required to deploy email notification via Amazon SNS for unsuppressed findings:

  1. Create and subscribe to a SNS topic
  2. Create an EventBridge rule

SNS is the target service for EventBridge actions in this workflow. SNS only requires two steps: 1. Create a new Topic (choose type as Standard) and provide a topic Name, 2. Create a Subscription with Email as the protocol. Entering an email address under Endpoint and clicking Create subscription finalizes the steps required in the console. The last step is to click Confirm subscription within the email sent to the email address used for the endpoint. GuardDuty SNS email Figure 10: GuardDuty SNS email SNS Subscription Confirmed

Figure 11: SNS subscription confirmation

The basis of EventBridge is to create rules that route events to a target. In this example below, a rule with an event pattern is constructed. This results in the rule being ran when an event matches the defined pattern. This rule will look for EC2 instances that exhibit SSH brute force attack behavior with the added detail of the resource being leveraged as the threat actor.

With both the SNS topic/subscription in place, deploying this EventBridge rule will automatically send an email notification to the registered email address. It is recommended to use an email alias associated with a team when sending notifications of high severity findings. GuardDuty EventBridge rule Figure 12: GuardDuty EventBridge rule

Tip: For more detailed instructions on how to build an EventBridge rule for GuardDuty findings that target SNS for email notification with customization, refer to this AWS Knowledge Center video.

Automated remediation for common finding types

Stealth:S3/ServerAccessLoggingDisabled A common use case for automation with GuardDuty finding types is to address S3 and EC2 related findings as a result of misconfiguration (intentional or unintentional). For example, finding type Stealth:S3/ServerAccessLoggingDisabled informs you that S3 server access logging is disabled for a bucket within your AWS environment. If disabled, no web request logs are created for any attempts to access the identified S3 bucket, however, S3 management API calls to the bucket, such as DeleteBucket, are still tracked. If S3 data event logging is enabled through CloudTrail for this bucket, web requests for objects within the bucket will still be tracked. This could be the result of a misconfiguration or part of an unauthorized user’s technique to evade detection.

Using the associated finding details, an administrator can learn about the AWS resources that were involved in the suspicious activity, when this activity took place, and additional information. After gathering the information, the administrator could pivot from the finding into the S3 bucket to re-enable Server Access Logging. Remediating this automatically would save the administrator time and allow for focused investigation into how a potential threat actor was able to perform the API call.

A common remediation approach decouples the GuardDuty detection from the automation action using an AWS Config Managed Rule to trigger on s3-bucket-logging-enabled configuration changes. Once triggered, AWS Config will immediately evaluate the S3 bucket and display a status of NON_COMPLIANT if logging is disabled. You can associate remediation actions with AWS Config rules and choose to execute them automatically to address non-compliant resources without manual intervention. In this scenario, the non-compliant s3-bucket-logging-enabled status will trigger the Systems Manager Automation document named AWS-ConfigureS3BucketLogging. By selecting Automatic remediation and AWS-ConfigureS3BucketLogging as your Remediation action detail when configuring your AWS Config rule Action, S3 server access logging will be re-enabled anytime a S3 bucket is non-compliant to the Config rule.

GuardDuty Finding automation workflow

Figure 13: GuardDuty finding automation

Backdoor:EC2/C&CActivity.B EC2 instances querying an IP address that is associated with a known command and control server is another common use case for automated remediation. This finding informs you that the listed instance within your AWS environment is querying an IP associated with a known command and control (C&C) server. The listed instance might be compromised. This type of finding has multiple steps towards remediations, that may include actions like performing a snapshot of the volume(s) before terminating the instance. This snapshot can provide information to a forensics team. However, the first step in remediating this finding is too isolate the instance.

The same event workflow described in the Automated Notifications section can be utilized to change the security group of the instance to stop inbound/outbound connections to the internet. An AWS Lambda function would be the target of an EventBridge rule set to match on Backdoor:EC2/C&CActivity.B. (An EventBridge can support multiple targets; a best practice for this scenario would also include an SNS topic target) Here is an example Lambda function written Python to perform this task:

import boto3

ec2_client = boto3.client('ec2')

def lambda_handler(event, context):
    try:
        # Check if the event is triggered by GuardDuty finding
        if 'detail' in event and 'type' in event['detail'] and event['detail']['type'] == 'Backdoor:EC2/C&CActivity.B':
            # Extract relevant information from the GuardDuty finding
            instanceId = event['detail']['resource']['instanceDetails']['instanceId']
            SecurityGroupId = "sg-XXXXXXXXXXXX"  # Replace with the desired Security Group ID

            # Construct the modification command
            command = {
                'InstanceId': instanceId,
                'Groups': [SecurityGroupId]
            }

            # Send the command to modify the EC2 instance's security group
            ec2_client.modify_instance_attribute(**command)

            return 'Updated security group for instance: {}'.format(instanceId)
        else:
            return 'No action taken. Event is not a GuardDuty finding of type Backdoor:EC2/C&CActivity.B'
    except Exception as e:
        print('Error:', str(e))
        raise e

Cost Optimization

When GuardDuty is enabled for the first time in an account, that account will be automatically provided with a 30-day free trial period for each region. Subsequently each feature also has a free trial too. For more information on pricing and free trials please refer to the GuardDuty pricing page. This is an important first step in understanding monthly GuardDuty costs in a given account. During the trial an administrator can view cost estimations based on Account ID, Data source, Feature and S3 buckets. If enabling GuardDuty in an AWS organization, the administrator can monitor cost metrics for all of the member accounts.

AWS account administrators may be required to investigate charges related to GuardDuty if an increase in costs occurs. Some common reason for increased monthly GuardDuty costs includes:

CloudTrail and/or S3 data event utilization is high

An increase in CloudTrail charges could be attributed to multiple reasons depending on whether they originate from management event costs or data event costs. Although there are many ways to query AWS log data we will show examples using Athena to look at exact API calls and the services associated with them. To understand CloudTrail API volumes using Athena:

  1. If not already present, create a new CloudTrail trail [2]. Choose to account for both management and data events here. If the trail is not needed beyond the scope of this analysis, the trail may be deleted or disable logging data to it.
  2. Create an Athena table [3] that pulls from the S3 location where the CloudTrail logs are stored
  3. Once this table has been created, navigate to the Athena console to perform a query over a 24-hour period to understand the top event names and event sources that contributed to the spike
SELECT eventName,count(eventName) AS eventVolume,eventSource
FROM your_athena_tablename
WHERE eventtime between '2021-01-24' AND '2021-01-25'
GROUP BY eventName, eventSource
ORDER BY eventVolume DESC limit 10;
  1. Upon gaining an understanding of which events and event sources were responsible for a bulk of the activity, it may be suitable to alter the SQL query to view data by hour, user-agent, IP or principal. This will help to identify the exact source of the increased API calls

[2] Creating a trail - https://docs.aws.amazon.com/awscloudtrail/latest/userguide/cloudtrail-create-a-trail-using-the-console-first-time.html [3] Querying AWS CloudTrail Logs - Using the CloudTrail Console to Create an Athena Table for CloudTrail Logs - https://docs.aws.amazon.com/athena/latest/ug/cloudtrail-logs.html#create-cloudtrail-table-ct

VPC Flow Logs

An increase in VPC Flow log related costs can be a result of GuardDuty analyzing flow logs to identify malicious, unauthorized, or unexpected behavior in AWS accounts and workloads. This data source is charged per Gigabyte (GB) per month and is offered with tiered volume discounts. To understand VPC Flow log volumes using Athena, perform the steps:

  1. Ensure that Flow logs are being published to S3 [4]
  2. Create an Athena table [5] that pulls from the S3 location where the flow logs are stored [5]
  3. Once this table has been created, navigate to the Athena console to perform a query over a 24 period to understand the top flows that contributed to the spike
SELECT COUNT(version) AS Flows,
        SUM(numpackets) AS Packetcount,
         sourceaddress
FROM vpc_flow_logs
WHERE date = DATE('2021-01-31')
GROUP BY  sourceaddress
ORDER BY  Flows DESC LIMIT 10;
  1. Alternatively, if VPC flow logs are setup with CloudWatch logs as their final destination, the customer may use CloudWatch insights to extract the top-talker information [6]. Here are some sample queries that can be called directly from the insights console [7]

[4] Publishing flow logs to Amazon S3 - https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-s3.html [5] Querying Amazon VPC Flow Logs - https://docs.aws.amazon.com/athena/latest/ug/vpc-flow-logs.html [6] Analyzing Log Data with CloudWatch Logs Insights - https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AnalyzingLogData.html [7] Sample Queries - https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/CWL_QuerySyntax-examples.html

Querying flow logs using Amazon Athena - https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs-athena.html*

DNS Query logs

An increase in DNS log related costs can be a result of GuardDuty analyzing DNS requests and responses to identify malicious, unauthorized, or unexpected behavior in AWS accounts and workloads. This data source is charged per Gigabyte (GB) per month and is offered with tiered volume discounts. Much like VPC Flow logs, an account may have their DNS flow logs exported to CloudWatch or S3 [8]. Top-talkers for DNS can be queried in CloudTrail Insights:

  1. Navigate to Route53 and verify that 'Query Logging' is enabled
  2. Check the logger configuration and confirm that a 'CloudWatch Logs log group' is the destination type
  3. Verify that one or more VPC’s are present under 'VPC’s that queries are logged for'
  4. Click the log group shown as a part of the 'Destination ARN'
  5. Click Insights on the left tab
  6. Select the log-group where the queries are being logged
  7. Select a time range and supply a query to understand which instance and hostnames have the most outbound queries
stats count(*) as numRequests by srcids.instance,query_name
    | sort numRequests desc
    | limit 10

Alternatively, if the DNS logs are being routed to a S3 bucket, Athena can be used to create a table that pulls this data from S3 [9]. Steps to perform:

  1. Navigate to Route53 and verify that 'Query Logging' is enabled
  2. Check the logger configuration and confirm that 'S3 bucket' is the destination type
  3. Verify that one or more VPC’s are present under 'VPC’s that queries are logged for'
  4. Navigate to Athena and create a table using the following command
CREATE EXTERNAL TABLE IF NOT EXISTS default.route53query (
  `version` float,
  `account_id` string,
  `region` string,
  `vpc_id` string,
  `query_timestamp` string,
  `query_name` string,
  `query_type` string,
  `query_class` string,
  `rcode` string,
  `answers` array<string>,
  `srcaddr` string,
  `srcport` int,
  `transport` string,
  `srcids` string,
  `isMatchedDomain` string
)
ROW FORMAT SERDE 'org.openx.data.jsonserde.JsonSerDe'
WITH SERDEPROPERTIES (
  'serialization.format' = '1'
) LOCATION 's3://<bucket name>/AWSLogs/<account number>/vpcdnsquerylogs/'
TBLPROPERTIES ('has_encrypted_data'='false');
  1. Query data using the following query
SELECT count(version) AS numRequests,
         srcids,
        query_name
FROM route53query
WHERE query_timestamp
    BETWEEN '2021-01-14'
        AND '2021-01-15'
GROUP BY  srcids,query_name
ORDER BY  numRequests DESC limit 10;

[8] Log your VPC DNS queries with Route 53 Resolver Query Logs - https://aws.amazon.com/blogs/aws/log-your-vpc-dns-queries-with-route-53-resolver-query-logs/ [9] How to automatically parse Route 53 Resolver query logs - https://aws.amazon.com/blogs/networking-and-content-delivery/how-to-automatically-parse-route-53-resolver-query-logs/ --

Other considerations

  • Sometimes an understanding of a surge in the costs occurring the start of the month for CloudTrail S3 and VPC/DNS flow log analysis is required. This is most likely due to the pricing structure of these data sources. GuardDuty offers tiered discounts and becomes inexpensive to use as more events are analyzed through the course of the month. See pricing page (https://aws.amazon.com/guardduty/pricing/) for a better understanding.
  • GuardDuty costs can increase due to configuration changes on an account including deployment of new applications or the use of a third-party security/scanning tool that makes repeated ‘Describe’ API calls. These might cause an increase in CloudTrail events, and/or traffic (VPC & DNS logs).
  • It is not recommended to rely on the CloudTrail event history's CSV download to compare event volumes because it may have event filters or incomplete data. This feature is more suited for smaller CSV downloads.
  • CloudTrail insights can be useful to augment these investigations. CloudTrail Insights continuously monitors CloudTrail write management events, and uses mathematical models to determine the normal levels of API and service event activity for an account.
  • If a new member account is added or if a GuardDuty service is resumed after being disabled for an extended time those costs may also be perceived as surges.
  • S3 protection can be disabled but GuardDuty does not allow you to remove any additional data sources. However, if DNS is disabled at the VPC level, those logs are not processed by GuardDuty. Accounts will not be charged or have DNS-based results.
  • GuardDuty is optimized for security value and will not charge customers for processing some low-risk events that would otherwise be delivered to a customer's CloudTrail. Therefore, the event counts may not exactly match.
  • If EKS Runtime Monitoring is enabled for your account, you will not be charged for analysis of VPC Flow Logs from instances where the GuardDuty agent is deployed and active.

Resources

Workshops

Videos

Blogs