DataOps Dashboard

Note: This documentation is also available in a rendered format here.

Creates CloudWatch dashboards for MDAA observability, aggregating metrics and logs from Lambda functions and other modules. Supports text, metric, and log insights widgets with SSM-based cross-module metric references and automatic widget layout. Use this module when you need a centralized view of your data pipeline health, including Lambda execution metrics, ETL job performance, and log-based error tracking.

Deployed Resources

This module deploys and integrates the following resources:

CloudWatch Dashboard(s) — One or more dashboards with configurable widget layouts. Widgets auto-wrap at 24-unit width.
SSM Parameter References — Resolves metric and log group references from other modules via SSM Parameter Store (e.g., Lambda function names, log groups, custom metric namespaces).

DataOps Project — Deploy the shared project infrastructure that dashboard metrics reference
Lambda Functions — Deploy Lambda functions whose metrics and logs can be visualized in dashboards
ETL Jobs — Deploy Glue ETL jobs whose metrics can be tracked in dashboards
Step Functions — Deploy Step Functions whose execution metrics can be monitored in dashboards

Security/Compliance Details

This module is designed in alignment with MDAA security/compliance principles and CDK nag rulesets. Additional review is recommended prior to production deployment, ensuring organization-specific compliance requirements are met.

Least Privilege:
- Dashboard access governed by IAM policies
- SSM parameter references use least-privilege read access to resolve cross-module metrics

Configuration

MDAA Config

Add the following snippet to your mdaa.yaml under the modules: section of a domain/env in order to use this module:

dataops-dashboard: # Module Name can be customized
  module_path: '@aws-mdaa/dataops-dashboard' # Must match module NPM package name
  module_configs:
    - ./dataops-dashboard.yaml # Filename/path can be customized

Module Config Samples and Variants

Copy the contents of the relevant sample config below into the ./dataops-dashboard.yaml file referenced in the MDAA config snippet above.

Minimal Configuration

Deploys a single CloudWatch dashboard with a text widget. Start here for a basic observability dashboard that you can incrementally add widgets to.

sample-config-minimal.yaml

# Contents available via above link
# Minimal DataOps Dashboard module configuration.
# Deploys a single CloudWatch dashboard with a text widget.

# (Optional) CloudWatch dashboard definitions.
dashboards:
  - dashboardName: my-dashboard
    widgets:
      - type: text
        markdown: '# My Dashboard'
        width: 24
        height: 2

Comprehensive Configuration

Demonstrates CloudWatch dashboards with text, metric, log insights, and advanced multi-metric widgets, all wired to a DataOps project. Start here when evaluating all available options for widget types, metric references, and cross-module SSM parameter resolution.

sample-config-comprehensive.yaml

# Contents available via above link
# Sample config for the DataOps Dashboard module.
# Demonstrates CloudWatch dashboards with text, metric, log insights,
# and advanced multi-metric widgets, all wired to a DataOps project.

# (Optional) DataOps project name for dashboard resource organization.
projectName: test-dashboard-project

# (Optional) SNS topic ARN for job notifications and workflow alerts. Auto-resolved from project
# when projectName is set.
notificationTopicArn: arn:{{partition}}:sns:{{region}}:{{account}}:test-topic

# (Optional) CloudWatch dashboard definitions for observability and monitoring. Dashboards can
# reference metrics from other modules via SSM parameters.
dashboards:
  # Dashboard with text widgets
  - dashboardName: test-text-dashboard
    widgets:
      - type: text
        markdown: |
          # Test Dashboard
          This is a test dashboard with text widgets
        width: 24
        height: 2

      - type: text
        markdown: '## Section 1'
        width: 12
        height: 2

      - type: text
        markdown: '## Section 2'
        width: 12
        height: 2

  # Dashboard with metric widgets using direct references
  - dashboardName: test-metric-dashboard
    widgets:
      - type: text
        markdown: '# Metrics Dashboard'
        width: 24
        height: 2

      - type: metric
        title: 'Error Count'
        width: 12
        height: 6
        metrics:
          - namespace: CustomMetrics
            metricName: ErrorCount
            stat: Sum

      - type: metric
        title: 'Duration'
        width: 12
        height: 6
        metrics:
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Average

  # Dashboard with log insights widgets
  - dashboardName: test-log-insights-dashboard
    widgets:
      - type: text
        markdown: '# Log Insights Dashboard'
        width: 24
        height: 2

      - type: log_insights
        title: 'Recent Errors'
        width: 24
        height: 6
        logGroupNames:
          - '/aws/lambda/test-function'
        queryString: |
          fields @timestamp, @message
          | filter @message like /ERROR/
          | sort @timestamp desc
          | limit 20

  # Dashboard with mixed widget types
  - dashboardName: test-mixed-dashboard
    widgets:
      - type: text
        markdown: '# Mixed Widget Dashboard'
        width: 24
        height: 2

      - type: metric
        title: 'Errors'
        width: 8
        height: 6
        metrics:
          - namespace: AWS/Lambda
            metricName: Errors
            stat: Sum

      - type: metric
        title: 'Invocations'
        width: 8
        height: 6
        metrics:
          - namespace: AWS/Lambda
            metricName: Invocations
            stat: Sum

      - type: metric
        title: 'Duration'
        width: 8
        height: 6
        metrics:
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Average

      - type: log_insights
        title: 'Recent Logs'
        width: 24
        height: 6
        logGroupNames:
          - '/aws/lambda/test-function'
        queryString: |
          fields @timestamp, @message
          | sort @timestamp desc
          | limit 50

  # Dashboard with advanced features (multiple metrics, metric math, period, dimensions)
  - dashboardName: test-advanced-dashboard
    widgets:
      - type: text
        markdown: |
          # 📊 Advanced Dashboard Features
          #### Testing multiple metrics per widget and metric math
          ---
        width: 24
        height: 3

      # Multiple metrics in one widget
      - type: metric
        title: 'Multiple Metrics - Error Counts'
        width: 12
        height: 6
        period: 300
        metrics:
          - namespace: ETL/ServiceA
            metricName: error-count
            stat: Sum
          - namespace: ETL/ServiceB
            metricName: error-count
            stat: Sum
          - namespace: ETL/ServiceC
            metricName: error-count
            stat: Sum

      # Widget with custom period
      - type: metric
        title: 'Duration (5 min period)'
        width: 12
        height: 6
        period: 300
        metrics:
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Average

      # Multiple metrics with different stats
      - type: metric
        title: 'Lambda Metrics - Mixed Stats'
        width: 24
        height: 6
        metrics:
          - namespace: AWS/Lambda
            metricName: Invocations
            stat: Sum
            label: 'Total Invocations'
          - namespace: AWS/Lambda
            metricName: Errors
            stat: Sum
            label: 'Total Errors'
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Average
            label: 'Avg Duration'
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Maximum
            label: 'Max Duration'

      # Metric with dimensions, unit, and per-metric period
      - type: metric
        title: 'Lambda by Function - Dimensions & Unit'
        width: 24
        height: 6
        metrics:
          # Metric with dimensions to filter by specific function
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Average
            label: 'Avg Duration (FuncA)'
            # Dimensions for the metric
            dimensions:
              FunctionName: test-function-a
            # Unit of the metric
            unit: Milliseconds
            # Period for the metric in seconds
            period: 60

      # Metric math expression widget
      - type: metric
        title: 'Error Rate (Metric Math)'
        width: 24
        height: 6
        metrics:
          # ID for the metric (used in expressions)
          - id: errors
            namespace: AWS/Lambda
            metricName: Errors
            stat: Sum
          - id: invocations
            namespace: AWS/Lambda
            metricName: Invocations
            stat: Sum
          # Metric math expression for calculations across metrics
          - id: error_rate
            expression: '100*(errors/invocations)'
            label: 'Error Rate %'

Standalone Configuration (No Project)

Demonstrates standalone CloudWatch dashboards with explicit KMS, bucket, deployment role, and security configuration instead of referencing a DataOps project. Use this when deploying outside of a DataOps project, providing infrastructure references directly.

sample-config-noproject.yaml

# Contents available via above link
# Sample config for the DataOps Dashboard module.
# Demonstrates standalone CloudWatch dashboards with explicit KMS,
# bucket, deployment role, and security configuration instead of
# referencing a DataOps project.

# (Optional) KMS key ARN for encrypting DataOps resources and data. Auto-resolved from project
# when projectName is set.
kmsArn: arn:{{partition}}:kms:{{region}}:{{account}}:key/test-key-id
# (Optional) S3 bucket name for project storage (scripts, artifacts, temp files). Auto-resolved
# from project when projectName is set.
bucketName: test-dashboard-bucket
# (Optional) IAM role ARN for deployment operations and resource management. Auto-resolved from
# project when projectName is set.
deploymentRoleArn: arn:{{partition}}:iam::{{account}}:role/test-deploy-role
# (Optional) Glue security configuration name for job encryption (at rest, in transit, CloudWatch
# logs). Auto-resolved from project when projectName is set.
securityConfigurationName: test-security-config
# (Optional) SNS topic ARN for job notifications and workflow alerts. Auto-resolved from project
# when projectName is set.
notificationTopicArn: arn:{{partition}}:sns:{{region}}:{{account}}:test-topic

# (Optional) CloudWatch dashboard definitions for observability and monitoring. Dashboards can
# reference metrics from other modules via SSM parameters.
dashboards:
  # Dashboard with text widgets
  - dashboardName: test-text-dashboard
    widgets:
      - type: text
        markdown: |
          # Test Dashboard
          This is a test dashboard with text widgets
        width: 24
        height: 2

      - type: text
        markdown: '## Section 1'
        width: 12
        height: 2

      - type: text
        markdown: '## Section 2'
        width: 12
        height: 2

  # Dashboard with metric widgets using direct references
  - dashboardName: test-metric-dashboard
    widgets:
      - type: text
        markdown: '# Metrics Dashboard'
        width: 24
        height: 2

      - type: metric
        title: 'Error Count'
        width: 12
        height: 6
        metrics:
          - namespace: CustomMetrics
            metricName: ErrorCount
            stat: Sum

      - type: metric
        title: 'Duration'
        width: 12
        height: 6
        metrics:
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Average

  # Dashboard with log insights widgets
  - dashboardName: test-log-insights-dashboard
    widgets:
      - type: text
        markdown: '# Log Insights Dashboard'
        width: 24
        height: 2

      - type: log_insights
        title: 'Recent Errors'
        width: 24
        height: 6
        logGroupNames:
          - '/aws/lambda/test-function'
        queryString: |
          fields @timestamp, @message
          | filter @message like /ERROR/
          | sort @timestamp desc
          | limit 20

  # Dashboard with mixed widget types
  - dashboardName: test-mixed-dashboard
    widgets:
      - type: text
        markdown: '# Mixed Widget Dashboard'
        width: 24
        height: 2

      - type: metric
        title: 'Errors'
        width: 8
        height: 6
        metrics:
          - namespace: AWS/Lambda
            metricName: Errors
            stat: Sum

      - type: metric
        title: 'Invocations'
        width: 8
        height: 6
        metrics:
          - namespace: AWS/Lambda
            metricName: Invocations
            stat: Sum

      - type: metric
        title: 'Duration'
        width: 8
        height: 6
        metrics:
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Average

      - type: log_insights
        title: 'Recent Logs'
        width: 24
        height: 6
        logGroupNames:
          - '/aws/lambda/test-function'
        queryString: |
          fields @timestamp, @message
          | sort @timestamp desc
          | limit 50

  # Dashboard with advanced features (multiple metrics, metric math, period)
  - dashboardName: test-advanced-dashboard
    widgets:
      - type: text
        markdown: |
          # 📊 Advanced Dashboard Features
          #### Testing multiple metrics per widget and metric math
          ---
        width: 24
        height: 3

      # Multiple metrics in one widget
      - type: metric
        title: 'Multiple Metrics - Error Counts'
        width: 12
        height: 6
        period: 300
        metrics:
          - namespace: ETL/ServiceA
            metricName: error-count
            stat: Sum
          - namespace: ETL/ServiceB
            metricName: error-count
            stat: Sum
          - namespace: ETL/ServiceC
            metricName: error-count
            stat: Sum

      # Widget with custom period
      - type: metric
        title: 'Duration (5 min period)'
        width: 12
        height: 6
        period: 300
        metrics:
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Average

      # Multiple metrics with different stats
      - type: metric
        title: 'Lambda Metrics - Mixed Stats'
        width: 24
        height: 6
        metrics:
          - namespace: AWS/Lambda
            metricName: Invocations
            stat: Sum
            label: 'Total Invocations'
          - namespace: AWS/Lambda
            metricName: Errors
            stat: Sum
            label: 'Total Errors'
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Average
            label: 'Avg Duration'
          - namespace: AWS/Lambda
            metricName: Duration
            stat: Maximum
            label: 'Max Duration'

Displays markdown content for dashboard headers, descriptions, and section dividers.

- type: text
  markdown: '# Dashboard Title'
  width: 24
  height: 2

Visualizes time-series metrics. Supports multiple metrics per widget, custom periods, stat types, and labels.

- type: metric
  title: 'Lambda Duration'
  width: 12
  height: 6
  period: 300
  metrics:
    - namespace: AWS/Lambda
      metricName: Duration
      stat: Average
      label: 'Avg Duration'

Displays CloudWatch Logs Insights query results.

- type: log_insights
  title: 'Recent Errors'
  width: 24
  height: 6
  logGroupNames:
    - '/aws/lambda/my-function'
  queryString: |
    fields @timestamp, @message
    | filter @message like /ERROR/
    | sort @timestamp desc
    | limit 20

Placeholders

{{function:functionName}} — Resolves Lambda function name from SSM
{{function:functionName:logGroup}} — Resolves Lambda function log group from SSM

Widgets auto-wrap at 24-unit width. When adding a widget would exceed 24 units, a new row starts automatically.

Config Schema Docs

DataOps Dashboard

Deployed Resources

Security/Compliance Details

Configuration

MDAA Config

Module Config Samples and Variants

Minimal Configuration

Comprehensive Configuration

Standalone Configuration (No Project)

Widget Types

Text Widget

Metric Widget

Log Insights Widget

Placeholders

Widget Layout

DataOps Dashboard

Deployed Resources

Related Modules

Security/Compliance Details

Configuration

MDAA Config

Module Config Samples and Variants

Minimal Configuration

Comprehensive Configuration

Standalone Configuration (No Project)

Widget Types

Text Widget

Metric Widget

Log Insights Widget

Placeholders

Widget Layout