Skip to content

SageMaker Project

Note: This documentation is also available in a rendered format here.

Deploys SageMaker Unified Studio projects, project profiles, and associated data sources for organizing and managing analytics workloads within a SageMaker domain. Use this module when you need to create governed analytics projects within SageMaker Unified Studio, with data sources imported from your Glue Catalog for team-based data exploration and publishing.

Note: SageMaker Projects can also be deployed via the DataOps Project module, allowing for automatic registration of Glue-based data sources and discovery of data assets. This module can be used when all data assets will be created and consumed entirely within SageMaker.


Deployed Resources

This module deploys and integrates the following resources:

Project Profiles - Enables blueprints for use in projects, with configurable environment templates.

Projects - SageMaker Unified Studio (DataZone V2) projects created within a domain.

Data Sources - Allows importing of existing Glue databases, or raw S3 locations, as data sources into SageMaker projects for publishing.

Metadata Form Types - Project-owned DataZone metadata form types, defined declaratively by their fields. MDAA assembles the Smithy model, so config authors do not write Smithy. See Metadata Form Types below for naming rules and the domain-redeploy prerequisite.

Glossaries - Project-owned DataZone business glossaries and terms, defined declaratively. A metadata form field's glossaryId can reference a declared glossary or term, resolved to the glossary's deploy-time ID. A term's relatesTo is wired by a separate custom resource (and its own Lambda handler) once every glossary/term in the project exists. See Glossaries below for the config surface and the domain-redeploy prerequisite.

Environment ID Resolution - For each environmentConfigs entry, a Lambda-backed custom resource (Custom::EnvironmentIdResolver) resolves the environment's live ID and publishes it to an SSM parameter at <ssmPath>/project/<key>-env-<configKey>/environment-id on every deploy, for use with the optional environmentId property. Setting environmentId is required to rename a project (or make any other change) once its environments are already ACTIVE - DataZone rejects such updates addressed by configuration name. See RENAMING_PROJECTS.md for details.

datazone


  • SageMaker (Domain) — Deploy the SageMaker domain that hosts projects created by this module
  • DataOps Project — Alternative way to deploy SageMaker projects with automatic Glue data source registration
  • Glue Catalog Settings — Configure cross-account Glue Catalog access for data sources imported into SageMaker projects
  • Lake Formation Access Control — Manage fine-grained Lake Formation grants for data sources used by SageMaker projects

Security/Compliance Details

This module is designed in alignment with MDAA security/compliance principles and CDK nag rulesets. Additional review is recommended prior to production deployment, ensuring organization-specific compliance requirements are met.

  • Encryption at Rest:
    • Projects inherit the domain-level customer-managed KMS encryption configuration
  • Least Privilege:
    • Projects are scoped to specific domain units and project profiles
    • Glue-based data sources use Lake Formation access control grants for fine-grained permissions on imported Glue databases
    • S3-based data sources grant the project's environment user role IAM read (list and get) on the registered bucket and prefix; the bucket policy and KMS key must also allow that role
    • Registering an S3 Access Grants location (registerS3AccessGrantLocation) is optional and additive

AWS Service Endpoints

When DataZone blueprints provision VPC-bound environments (e.g., SageMaker Studio, Athena, Glue), the following VPC endpoints may be required if public AWS service endpoint connectivity is unavailable (e.g., private subnets without NAT gateway, firewalled environments, or PrivateLink-only architectures). Specific endpoints depend on which blueprints are enabled:

AWS Service Endpoint Service Name Type
DataZone com.amazonaws.{region}.datazone Interface
SageMaker API com.amazonaws.{region}.sagemaker.api Interface
SageMaker Runtime com.amazonaws.{region}.sagemaker.runtime Interface
SageMaker Studio com.amazonaws.{region}.studio Interface
Athena com.amazonaws.{region}.athena Interface
Glue com.amazonaws.{region}.glue Interface
Lake Formation com.amazonaws.{region}.lakeformation Interface
KMS com.amazonaws.{region}.kms Interface
S3 com.amazonaws.{region}.s3 Gateway
CloudWatch Logs com.amazonaws.{region}.logs Interface
STS com.amazonaws.{region}.sts Interface
EFS com.amazonaws.{region}.elasticfilesystem Interface

Configuration

MDAA Config

Add the following snippet to your mdaa.yaml under the modules: section of a domain/env in order to use this module:

sagemaker-project: # Module Name can be customized
  module_path: '@aws-mdaa/sagemaker-project' # Must match module NPM package name
  module_configs:
    - ./sagemaker-project.yaml # Filename/path can be customized

Data Sources

A project's dataSourceDefaults supplies publishOnImport, enableSetting, schedule, recommendation, autoImportDataQualityResult, and filterExpressions values inherited by every Glue database (databaseName) entry in dataSources; an s3 entry has no equivalent AWS::DataZone::Connection properties, so these defaults are silently not applied to it, and setting one of these fields directly on an s3 entry itself (rather than inheriting it) fails synth with a clear error. A Glue database data source that sets one of these fields itself replaces the corresponding default entirely for that field — fields are not deep-merged, so a data source overriding schedule must supply the whole schedule object, not just the part that differs. There is no way for a data source to opt out of an inherited schedule other than setting its own enableSetting to DISABLED, which disables the whole data source.

In the DataOps Project module's SMUS (sagemaker) mode, sagemaker.project.dataSourceDefaults also applies to the project's own database-backed data sources (databases.<db>.createSagemakerDatasource/createDatazoneDatasource), since those share this module's SageMakerProjectProps type; see that module's README.

A Glue database data source's metadataForms attaches the project's own metadataForms (declared under the same project, in any order relative to dataSources) to every asset the data source imports, pre-populating form field values via content. The map key must match a key declared under the project's metadataForms and reference an ENABLED form; content, if set, must be valid JSON naming only fields the form declares, and every field the form declares required: true must be present (via content or otherwise). Any violation fails synth with a clear error. metadataForms only applies to Glue database data sources (databaseName) — setting it on an s3 data source fails synth, since an AWS::DataZone::Connection has no equivalent property.

Metadata Form Types

Form and field names must be valid Smithy identifiers (letter or underscore first, then letters, digits, or underscores), and form names must be unique across the domain. Deleting a form type on stack teardown is best-effort: if DataZone rejects the delete (for example, because the form is still attached to assets), the form type is left in place. Existing deployments must redeploy the DataZone/SageMaker domain module before deploying a project with metadataForms, so that the domain grants form-type creation to the custom-resource role.

Glossaries

The glossary key is used verbatim as the DataZone glossary name, which must be unique across the domain, not just the project. A field's glossaryId may reference a declared glossary under the project's glossaries as "<glossaryName>", or a declared term as "<glossaryName>.<termName>", resolved to that glossary's deploy-time ID; or a literal, pre-existing glossary ID string, unchanged. On teardown, MDAA deletes the declared terms, then disables and deletes the glossary. Both deletes are best-effort: if DataZone rejects one, the resource is left in place and the stack update still succeeds. If a leftover glossary later blocks recreating one with the same name, delete the existing glossary directly in DataZone, or, if you are moving a glossary between projects, remove it from glossaries and deploy, then add it back. A glossary or term can't move between projects or domains: Update detects this and fails clearly rather than attempting an unsupported move. Existing deployments must redeploy the DataZone/SageMaker domain module before deploying a project with glossaries, so that the domain grants glossary creation to project owners.

A term's description is its short, business-facing definition; its readme is longer-form Markdown documentation (calculation methodology, data sources, usage guidelines, etc.), shown as the term's README in SageMaker Unified Studio. A term's relatesTo lists other declared terms it "is a type of" (DataZone's termRelations.isA), each a bare "<termName>" for a term in the same glossary or "<glossaryName>.<termName>" for a term in another glossary declared by the same project, up to 10 entries; SageMaker Unified Studio shows the inverse "Has Types" relationship on the related term automatically. A name resolves only against glossaries declared under this project; anything else must be a literal, pre-existing term ID, and a cycle of relatesTo references across declared terms fails synth with a clear error.

Module Config Samples and Variants

Copy the contents of the relevant sample config below into the ./sagemaker-project.yaml file referenced in the MDAA config snippet above.

Minimal Configuration

Deploys a single SageMaker project with a project profile. Start here for a basic SageMaker project within an existing domain.

sample-config-minimal.yaml

# Contents available via above link
# Minimal SageMaker Project module configuration.
# Deploys a single SageMaker project with a project profile.

# SSM parameter for SageMaker Domain config resolution
# Often created by the SageMaker module.
# Example SSM: ssm:/{{org}}/{{domain}}/<sagemaker_module_name>/domain/<domain_name>/config
domainConfigSSMParam: /test-org/test-domain/test-module/domain/test-sus-domain/config

# (Optional) Project profiles — reusable templates that determine
# which environments are provisioned when a project is created.
projectProfiles:
  my-profile:
    environments:
      my-env:
        deploymentMode: ON_CREATE

# (Optional) SageMaker projects — the module's primary resource.
projects:
  my-project:
    # Name of the project profile to use
    profileName: my-profile

Comprehensive Configuration

Covers both ON_CREATE and ON_DEMAND deployment modes, environment templates, project profiles with all options, and projects with full membership. Use this as a reference when you need full control over deployment modes, environment templates, data sources, and project membership.

sample-config-comprehensive.yaml

# Contents available via above link
# Comprehensive sample config for the SageMaker Project module.
# Exercises ALL compatible non-excluded properties at full depth.
# Covers both deploymentMode enum variants, environment templates,
# project profiles with all options, and projects with full membership.
#
# Mutually exclusive: domainConfigSSMParam vs domainConfig.
# domainConfig is a runtime-resolved CDK construct (not user-configurable YAML).
# Use domainConfigSSMParam for SSM-based domain config resolution.

# (Optional) SSM parameter base name for the SageMaker Domain config.
# Resolves domain ID, blueprint IDs, domain unit IDs from SSM and APIs.
# Mutually exclusive with domainConfig.
# Often created by the SageMaker module.
# Example SSM: ssm:/{{org}}/{{domain}}/<sagemaker_module_name>/domain/<domain_name>/config
domainConfigSSMParam: /test-org/test-domain/test-module/domain/test-sus-domain/config

# (Optional) Reusable environment templates referenced by project profiles
# via the environmentsTemplate property. Template environments are merged
# with profile-specific environments.
projectProfileEnvironmentsTemplates:
  # Template with ON_CREATE deployment and parameter overrides
  test-template-on-create:
    # Environment name → ProfileEnvironmentConfig
    test-env-blueprint:
      # (Optional) Deployment mode enum: ON_CREATE | ON_DEMAND
      deploymentMode: ON_CREATE
      # (Optional) Numeric deployment order; lower deploys first
      deploymentOrder: 1
      # (Optional) Blueprint parameter overrides
      parameters:
        overrides:
          param-one:
            # (Optional) Override value for this blueprint parameter
            value: override-value-one
            # (Optional) Whether project creators can edit this parameter
            isEditable: true
          param-two:
            value: locked-value
            isEditable: false

  # Template with ON_DEMAND deployment (second enum variant)
  test-template-on-demand:
    test-env-blueprint-demand:
      deploymentMode: ON_DEMAND
      deploymentOrder: 2
      parameters:
        overrides:
          demand-param:
            value: demand-override
            isEditable: false

# (Optional) Project profiles defining environment blueprints and
# deployment configurations. Profiles are reusable templates that
# determine which environments are provisioned when a project is created.
projectProfiles:
  # Profile using an environment template with all optional properties
  test-profile-full:
    # (Optional) Target AWS account ID for the profile's environments
    account: '{{account}}'
    # (Optional) Target AWS region for the profile's environments
    region: '{{region}}'
    # (Optional) Domain unit path for profile scoping
    domainUnit: /root/team-a
    # (Optional) Reference to a template in projectProfileEnvironmentsTemplates
    environmentsTemplate: test-template-on-create
    # (Optional) Profile-specific environments merged with template environments
    environments:
      profile-specific-env:
        deploymentMode: ON_CREATE
        deploymentOrder: 3
        parameters:
          overrides:
            profile-param:
              value: profile-value
              isEditable: true

  # Minimal profile with only environments (no template, no account/region)
  test-profile-minimal:
    environments:
      minimal-env:
        deploymentMode: ON_DEMAND

# (Optional) SageMaker projects to create in the domain. Each project
# references a project profile and can include data sources and membership.
projects:
  # Project with all optional properties exercised
  test-project-full:
    # (Required) Name of the project profile to use
    profileName: test-profile-full
    # (Optional) Exact DataZone project name used verbatim, bypassing MDAA naming.
    # Free-form business label for the SMUS project title; 1-64 chars matching
    # ^[\w -]+$ (spaces and mixed case allowed). All other resources keep MDAA naming.
    nameOverride: Customer 360 Analytics
    # (Optional) Domain unit path where the project will be created
    domainUnit: /some/domain/unit
    # (Optional) Per-environment configuration overrides
    environmentConfigs:
      test-env-blueprint:
        # (Optional) Map of parameter name to value
        parameters:
          env-param-key: env-param-value
      # (Optional) Apply this entry to a specific environment ID instead of looking it up by name
      profile-specific-env:
        environmentId: test-environment-id
        parameters:
          profile-param: pinned-value
    # (Optional) Default values for publishOnImport, enableSetting, schedule, recommendation,
    # autoImportDataQualityResult, and filterExpressions, inherited by every entry in
    # dataSources. A data source that sets one of these fields itself overrides the default
    # for that field entirely (fields are not deep-merged); see test-source below.
    dataSourceDefaults:
      # (Optional) Auto-publish imported assets to the project's catalog
      publishOnImport: true
      # (Optional) Enabled setting enum: ENABLED | DISABLED
      enableSetting: ENABLED
      # (Optional) Schedule the data source runs on
      schedule:
        # (Required) Cron expression for the run schedule
        schedule: cron(0 6 * * ? *)
        # (Optional) Timezone the schedule is evaluated in
        timezone: AMERICA_MONTREAL
      # (Optional) Business name generation recommendation
      recommendation:
        # (Optional) Enable automatic business name generation
        enableBusinessNameGeneration: true
    # (Optional) Data sources to import into the project
    dataSources:
      test-source:
        # (Required) Glue database name for the data source. Mutually exclusive with s3.
        databaseName: test-database-name
        # (Optional) Overrides dataSourceDefaults.publishOnImport for this source only
        publishOnImport: false
        # (Optional) Whether to auto-import AWS Glue Data Quality results
        autoImportDataQualityResult: false
        # (Optional) Limits imported tables to those matching these filter expressions.
        # Controls catalog import only, not access: Lake Formation still grants read on
        # every table in the database regardless of these filters.
        filterExpressions:
          - type: INCLUDE
            expression: dim_*
          - type: EXCLUDE
            expression: tmp_*
        # (Optional) Metadata forms (declared under this project's metadataForms, below) to
        # attach to every asset this data source imports
        metadataForms:
          CustomerMetadataForm:
            # (Optional) JSON-encoded field values to pre-populate on the form
            content: '{"customerName":"unknown"}'
      second-source:
        # Inherits dataSourceDefaults in full: no fields overridden here
        databaseName: another-database
      s3-source:
        # (Required) S3 location to register as a data source. Mutually exclusive with databaseName.
        s3:
          # (Required) Bucket name, or an ssm-org:/ssm-domain: reference to a bucket created by another module
          bucketName: test-data-bucket
          # (Optional) Key prefix within the bucket to register; registers the whole bucket when omitted
          prefix: raw/
    # (Optional) Owner users with PROJECT_OWNER designation
    ownerUsers:
      owner1: test-owner-user
    # (Optional) Owner groups with PROJECT_OWNER designation
    ownerGroups:
      ownergrp1: test-owner-group
    # (Optional) Contributor users with PROJECT_CONTRIBUTOR designation
    users:
      user1: test-contributor-user
    # (Optional) Contributor groups with PROJECT_CONTRIBUTOR designation
    groups:
      grp1: test-contributor-group
    # (Optional) Business glossaries owned by this project. A metadata form field's
    # glossaryId may reference a declared glossary here as "<glossaryName>", or a
    # declared term as "<glossaryName>.<termName>", resolved by MDAA to the glossary's
    # deploy-time ID, or continue to use a literal, pre-existing glossary ID string
    # (see tier.glossaryId below).
    glossaries:
      # Glossary exercising all term options
      CustomerGlossary:
        # (Optional) Description of the glossary
        description: Standard customer business terms
        # (Optional) Status enum: ENABLED | DISABLED
        status: ENABLED
        # (Optional) Term name to GlossaryTermProps
        terms:
          Segment:
            # (Optional) Short, business-facing definition
            description: Customer segment classification
            # (Optional) Longer-form Markdown documentation, shown as the term's README in
            # SageMaker Unified Studio
            readme: The marketing segment a customer has been classified into
            status: ENABLED
          HighValueSegment:
            description: A customer segment with disproportionately high lifetime value
            # (Optional) Other declared terms this term "is a type of". A bare name
            # references a term in this same glossary; "<glossaryName>.<termName>"
            # references a term in another glossary declared by this project. A
            # HighValueSegment is a more specific kind of Segment.
            relatesTo:
              - Segment
      # Glossary demonstrating a cross-glossary relatesTo reference into CustomerGlossary
      ComplianceGlossary:
        terms:
          AuditedSegment:
            description: A customer segment that has passed internal compliance review
            # An AuditedSegment is a more specific kind of HighValueSegment, declared
            # under a different glossary than the term it relates to.
            relatesTo:
              - CustomerGlossary.HighValueSegment
    # (Optional) Metadata form types owned by this project. MDAA assembles the
    # Smithy model; the form name is used verbatim as the structure name.
    metadataForms:
      # Form exercising all field options
      CustomerMetadataForm:
        # (Optional) Description of the form type
        description: Customer governance metadata
        # (Optional) Status enum: ENABLED | DISABLED
        status: ENABLED
        # (Required) Field name → FormFieldProps
        fields:
          # Required, technical-identifier searchable string
          customerName:
            # (Required) Smithy prelude type
            type: String
            # (Optional) Annotates @documentation, shown to stewards filling out the form
            description: Unique identifier for the customer account
            # (Optional) Annotates @required
            required: true
            # (Optional) @amazon.datazone#searchable modes
            searchable:
              - TECHNICAL
          # Lexical searchable string tied to a pre-existing, manually-created glossary
          tier:
            type: String
            searchable:
              - LEXICAL
            # (Optional) @amazon.datazone#glossaryterm(<id>); a literal, pre-existing glossary ID
            glossaryId: test-glossary-id
          # String field linked to CustomerGlossary, referenced through its declared Segment term
          customerSegment:
            type: String
            # (Optional) "<glossaryName>.<termName>" resolves to the glossary's deploy-time ID
            glossaryId: CustomerGlossary.Segment
          # Non-string field types
          activeFlag:
            type: Boolean
          recordCount:
            type: Integer
          createdAt:
            type: Timestamp
      # Minimal form with a few fields, defaulted ENABLED status.
      # Factored into its own file via !includeYaml, parsed as YAML and spliced in
      # directly (unlike !include, which inlines a referenced file as a raw string).
      SimpleForm: !includeYaml ./simple-metadata-form.yaml

  # Minimal project with only required profileName
  test-project-minimal:
    profileName: test-profile-minimal

The comprehensive sample's SimpleForm metadata form is factored out into its own file and pulled in via !includeYaml. This file must sit next to the module config (i.e. in the same directory as sagemaker-project.yaml) for the reference to resolve.

simple-metadata-form.yaml

# Contents available via above link
# Metadata form definition included via !includeYaml from sample-config-comprehensive.yaml.
# This file is a whole YAML document, but not a full module config - its content is
# parsed as YAML and spliced in at the point of inclusion, so it is written exactly as
# it would appear inline (a MetadataFormProps value).
# (Required) Field name → FormFieldProps
fields:
  note:
    # (Required) Smithy prelude type
    type: String
    # (Optional) Annotates @documentation, shown to stewards filling out the form
    description: Free-form note about this asset
  priority:
    type: Integer
    # (Optional) Annotates @required
    required: true
  reviewedAt:
    type: Timestamp

Config Schema Docs