Testing
MDAA employs a layered testing strategy that mirrors the construct architecture. Each layer has distinct testing goals, tools, and coverage expectations.
Standards
- All packages require 80% branch and 80% statement coverage, enforced via the root
jest.config.js - App packages inherit a lower branch threshold (0%) via
packages/apps/jest.config.jsdue to an Istanbul/ts-jest quirk with default constructor parameters - All compliance controls must have explicit test assertions, not just coverage
- CDK Nag rulesets (AwsSolutions, NIST 800-53 R5, HIPAA Security, PCI DSS 3.2.1) are validated in construct tests via
MdaaTestApp.checkCdkNagCompliance() - Tests run with
jest --passWithNoTests --coverageas a single unified command - Diff baselines are committed to the repository and reviewed as part of code changes
- Non-deterministic test values use
test-account,test-region,test-partitionfor stable, reproducible output - The CLI is covered by command baselines — golden shell command strings — because its output is a command, not a template
Quick Reference
Running Tests
# From the repository root:
npm test # Affected TS tests + all Python tests
npm run test:all # All TS tests, no cache
npm run test:python # Python tests on affected packages
npm run test:python:all # Python tests on all packages
npm run test:update-baselines # Regenerate diff baselines after intentional changes
npm run test:starter-kits # Starter kit tests, scoped to what the change affects
npm run test:starter-kits:all # All starter kit tests for all kits
# From any package directory:
npm test # Run that package's tests with coverage
Starter kit tests are a separate command because they run the real CLI end to end per kit; npm test does not include them.
Pre-push Validation
Run the MR pipeline checks locally before pushing:
npm run prepush # Affected only, uses Nx cache (fast)
npm run prepush:all # All packages, no cache (thorough)
Both run the same 6 steps, differing only in scope:
- Validate package structure (
validate_packages.sh) - Validate dependency lock file (
validate_dependencies.sh) - Lint TypeScript
- Lint Python
- Prettier
- Build + test
Force Commands (no Nx affected, no cache)
npm run build:all # Build all packages
npm run test:all # Test all packages
npm run lint:all # Lint all packages
npm run prettier:all # Prettier all packages
Jest Configuration
All 132 package jest configs inherit from the root jest.config.js:
jest.config.js (root) ← shared defaults + setupFiles
├── packages/apps/jest.config.js ← apps base (branch threshold: 0%)
│ └── ~30 app package configs ← inherit from apps base
└── ~102 construct/utility/cli configs ← inherit from root directly
The root config provides:
- roots, testMatch, transform (ts-jest), coverageReporters
- setupFiles → jest.setup.js (mocks Docker/pip builds globally)
- coverageThreshold: 80% branches, 80% statements
Per-package configs only override what differs (typically coverageThreshold). No standalone configs exist.
Global Test Mocks (jest.setup.js)
The root jest.setup.js runs before every test file in every package. It prevents Docker and pip builds during tests:
- Mocks
command-exists→syncreturnsfalse(no Docker/finch detection) - Stubs
Code.fromDockerBuildandCode.fromCustomCommand→ return mock code objects - Preserves all other
Codestatic methods (fromAsset,fromInline, etc.)
This eliminates the need for per-test-file Docker mocks. Three test files retain their own command-exists mocks because they test behavior that depends on it:
- code-asset.compliance.test.ts — tests the Docker vs pip branching logic
- mdaa-cli.test.ts / mdaa-cli-sanity.test.ts — CLI tests that simulate Docker availability
L2 Constructs
L2 constructs wrap CDK L1/L2 constructs with compliance controls and developer experience improvements such as standardized props typing, MDAA naming conventions, and automatic CDK Nag suppression management.
What to Test
Every compliance-related feature must be tested:
- Resource property enforcement (encryption, logging, access controls)
- CDK Nag compliance (AwsSolutions, NIST 800-53, HIPAA, PCI DSS)
- MDAA naming convention application
- SSM parameter and CloudFormation output generation
- IAM policy scoping and least-privilege enforcement
- Input validation and error handling
How to Test
L2 tests use the CDK Assertions library (Template.fromStack()) to inspect synthesized CloudFormation templates. Tests instantiate the construct with MdaaTestApp, then assert on resource properties, resource counts, and CDK Nag compliance.
import { MdaaTestApp } from '@aws-mdaa/testing';
import { Match, Template } from 'aws-cdk-lib/assertions';
import { MdaaKmsKey } from '../lib';
describe('MDAA Compliance Tests', () => {
const testApp = new MdaaTestApp();
new MdaaKmsKey(testApp.testStack, 'test-key', {
naming: testApp.naming,
alias: 'test-key',
keyUserRoleIds: ['test-user-id'],
keyAdminRoleIds: ['test-admin-id'],
});
testApp.checkCdkNagCompliance(testApp.testStack);
const template = Template.fromStack(testApp.testStack);
test('Key rotation is enabled', () => {
template.hasResourceProperties('AWS::KMS::Key', {
EnableKeyRotation: true,
});
});
});
File Naming
{construct}.compliance.test.tsfor compliance and property assertions{construct}.test.tsfor additional unit tests (edge cases, error handling)
Coverage
L2 constructs require 80% branch and statement coverage. All compliance controls must have explicit test assertions.
L3 Constructs
L3 constructs implement architectural patterns and multi-resource integrations. They compose L2 constructs and CDK resources into higher-level abstractions (e.g., a data lake with buckets, encryption, access controls, and Lake Formation permissions).
L3 constructs also implement compliance controls when a reusable L2 construct isn't warranted for a particular resource type.
What to Test
- All compliance controls implemented at the L3 level (encryption, IAM policies, security groups, logging)
- Resource composition and dependency ordering
- Constructor input validation and error handling
- Cross-account and cross-region resource generation
- CDK Nag compliance for the full construct tree
How to Test
Same approach as L2: CDK Assertions library with MdaaTestApp. L3 tests typically have more complex setup because they compose multiple resources.
import { MdaaTestApp } from '@aws-mdaa/testing';
import { MdaaRoleHelper } from '@aws-mdaa/iam-role-helper';
import { Template } from 'aws-cdk-lib/assertions';
import { MyL3Construct } from '../lib';
describe('MDAA Compliance Tests', () => {
const testApp = new MdaaTestApp();
new MyL3Construct(testApp.testStack, 'test-construct', {
naming: testApp.naming,
roleHelper: new MdaaRoleHelper(testApp.testStack, testApp.naming),
// ... construct-specific props
});
testApp.checkCdkNagCompliance(testApp.testStack);
const template = Template.fromStack(testApp.testStack);
test('S3 bucket has encryption', () => {
template.hasResourceProperties('AWS::S3::Bucket', {
BucketEncryption: {
/* ... */
},
});
});
});
File Naming
{construct}.compliance.test.tsfor compliance and resource property assertions{construct}.test.tsfor functional unit tests{construct}.constructor.test.tsorconstructor-exceptions.test.tsfor input validationvalidators.test.tsfor standalone validation logic
Coverage
L3 constructs require 80% branch and statement coverage. All compliance controls must have explicit test assertions, whether implemented at the L3 level or delegated to L2 constructs.
Apps / Modules
App modules implement configuration schemas (TypeScript interfaces auto-generated to JSON Schema) and translate user-provided YAML configuration into L3 construct props. They are the user-facing entry point for deploying infrastructure.
What to Test
App-level testing validates the full pipeline from configuration to CloudFormation output:
- Configuration schema coverage: every schema property exercised through sample configs
- Schema validation: invalid configs rejected, required fields enforced
- Infrastructure stability: generated CloudFormation resources do not change unexpectedly across versions
- Multi-config variant coverage: mutually exclusive config branches each have dedicated sample configs
How to Test: Diff Baseline Testing
App modules use CDK diff-based baseline testing. This approach uses the CDK toolkit's semantic diff engine to compare synthesized CloudFormation templates against committed baselines.
Each sample config gets a baselineDiffTestApp call that:
- Synths the CDK app with the sample config
- Stores the CloudFormation template as a
.baseline.jsonfile - On subsequent runs, diffs the current output against the baseline using
@aws-cdk/toolkit-lib - Fails if resources or outputs changed
Only resource and output differences are flagged. Metadata changes (CDK Nag suppression annotations, asset hashes, CDK version metadata) are ignored, eliminating false positives that plagued traditional snapshot testing.
import { describe } from '@jest/globals';
import { baselineDiffTestApp, Create } from '@aws-mdaa/testing';
import { MyModuleApp } from '../lib/my-module';
import * as path from 'path';
describe('MyModule Baseline Diff Tests', () => {
baselineDiffTestApp(
'MyModule Comprehensive',
Create.appProvider(
context => {
const moduleApp = new MyModuleApp({
context: {
...context,
module_configs: path.join(__dirname, '..', 'sample_configs', 'sample-config-comprehensive.yaml'),
},
});
moduleApp.generateStack();
return moduleApp;
},
{
module_name: 'test-my-module-app',
org: 'test-org',
env: 'test-env',
domain: 'test-domain',
},
),
);
});
Sample Configs
Every app module has sample configuration files under sample_configs/:
sample-config-minimal.yamlfor the simplest valid deploymentsample-config-comprehensive.yamlexercising every compatible schema propertysample-config-{variant}.yamlfor mutually exclusive configuration branches
Sample configs use template variables ({{region}}, {{account}}, {{partition}}) instead of hard-coded AWS values. Cross-account references use {{context:account-2}}, {{context:account-3}}.
Each sample config must have a corresponding baselineDiffTestApp call in the module's diff test file.
Baseline Files
Baselines are stored as JSON in test/__snapshots__/:
- Single-stack:
{configBaseName}.baseline.json - Multi-stack:
{configBaseName}.{stackName}.baseline.json
Baselines are committed to the repository. When infrastructure changes are intentional:
# From the repo root — update baselines for affected packages
npm run test:update-baselines
# From a specific package directory
UPDATE_BASELINES=true npm test
Review the diff output before committing updated baselines to confirm changes are intentional.
Handling Non-Deterministic Resources
Some resources produce non-deterministic output (e.g., timestamps, generated IDs). Two options are available:
Strip entire resources — use ignoreResourcePatterns when the entire resource is non-deterministic (e.g., Lambda versions with hash-based logical IDs):
baselineDiffTestApp('MyModule Comprehensive', appProvider, {
ignoreResourcePatterns: ['scheduledaction'],
});
Strip specific properties — use ignoreResourceProperties when a resource is mostly stable but has one or two non-deterministic properties (e.g., a refresh timestamp). This keeps the resource in the baseline so its other properties are still validated:
baselineDiffTestApp('MyModule Comprehensive', appProvider, {
ignoreResourceProperties: { 'domainConfigcr': ['refresh'] },
});
Both options strip the ignored content from the baseline template at write time, so baselines only contain validated content.
File Naming
{module-name}.diff.test.tsfor diff baseline tests (one per module)test/__snapshots__/*.baseline.jsonfor committed baseline templates
Coverage
App modules require 80% statement coverage. Branch coverage is set to 0% at the apps base level due to an Istanbul/ts-jest quirk where default constructor parameters (props: AppProps = {}) count as uncovered branches even when all real logic is tested.
Diff Risk Assessment
Baseline diffs must be reviewed carefully before committing. Not all diffs are equal — some represent routine changes, while others can cause data loss or deployment failures in existing environments. Treat every baseline update as a change to deployed infrastructure.
Breaking Diffs
A breaking diff is any change that would cause a CloudFormation deployment failure in an existing environment. Breaking diffs must be avoided at all costs. Common causes:
- Renaming a CloudFormation logical ID for a stateful resource (S3 bucket, DynamoDB table, RDS instance) — CloudFormation deletes the old resource and creates a new one, losing all data
- Changing a resource property that requires replacement (e.g.,
BucketNameon an S3 bucket,TableNameon a DynamoDB table) — same delete-and-recreate behavior - Removing a resource that other stacks or external systems depend on (exports, SSM parameters, IAM roles referenced by running workloads)
Data-Containing Resource Diffs
Any diff that deletes or replaces a data-containing resource requires explicit justification and review. These resources include S3 buckets, DynamoDB tables, RDS/Aurora instances, OpenSearch domains, EFS file systems, and Glue databases/tables. Even if the replacement resource is identical, the data in the original resource is lost.
When reviewing a baseline diff, flag any resource where Action: DELETE or Action: REPLACE appears on a data-containing resource type. If the change is intentional, document why data loss is acceptable or how data migration will be handled.
Construct ID and Scoping Changes
CDK generates CloudFormation logical IDs from the construct tree path. Changes to construct IDs, nesting depth, or scope hierarchy silently change logical IDs, which CloudFormation interprets as "delete old resource, create new one." This is particularly dangerous because:
- The new resource has the same configuration as the old one, so the diff looks harmless — just a logical ID rename
- But CloudFormation will attempt to create the new resource before deleting the old one, causing naming collisions if the resource has a fixed physical name (e.g., S3 bucket name, IAM role name, SSM parameter path)
- Even if creation succeeds, the old resource is orphaned or deleted, losing any data it contained
Watch for these patterns in diffs:
- A resource disappearing at one logical ID and appearing at another with identical properties
- Changes to construct
idparameters in L2/L3 constructors - Moving a construct from one scope to another (e.g., from the stack directly into a nested construct)
- Refactoring that changes the construct tree depth (adding or removing intermediate constructs)
When a logical ID change is unavoidable, it must be handled with a CloudFormation resource import or a migration plan — never by simply updating the baseline and deploying.
Privilege Escalation Diffs
Changes that increase permissions or broaden access require security review, even though they deploy successfully. Watch for:
- IAM policy statements with added actions, especially
*actions or sensitive actions (iam:PassRole,sts:AssumeRole,kms:Decrypt) - Removal of IAM policy conditions that previously scoped access (
aws:SourceArn,aws:SourceAccount) - Broadened resource ARNs (e.g., scoped prefix → wildcard)
- New principals in resource policies (S3, KMS, SNS/SQS)
- Security group ingress additions, especially
0.0.0.0/0 - Removal of
DeletionPolicy: RetainorUpdateReplacePolicy: Retain
Privilege escalation is dangerous because it deploys without errors but silently weakens the security posture. Verify the increase is intentional, justified, and follows least-privilege principles.
New CDK Nag Suppressions
New NagPackSuppression entries — whether in construct code or config-level nag_suppressions — are explicit opt-outs from compliance controls. Each new suppression requires compliance review:
- The reason must be specific and reference AWS service authorization documentation
- The suppression must be scoped as narrowly as possible (resource-level, not stack-level)
- Suppressions that bypass encryption or access control requirements are highest priority for review
Kiro: The
diff-risk-assessmentsteering file automates reviewing baseline diffs for breaking changes, data loss risks, and construct ID scoping issues. It activates automatically when baseline files are modified.
CLI
The CLI (packages/cli) resolves the config hierarchy, transforms {{...}} references, and interpolates the result into one shell command per module, which it then executes in a child process. Its observable contract is therefore the command string, not a CloudFormation template — every step runs before any template exists.
Template baselines cannot cover this. App-level diff tests build their CDK context in-process, so they never exercise the encode-to-argv step where quoting, escaping, and reference interpolation happen. A change to command assembly leaves every template baseline passing.
How to Test: CLI Command Baseline Testing
Sample configs run through the real CLI in --testing mode, which prints each command instead of executing it. No AWS credentials, CDK, network, Terraform, Checkov, or pip is needed, so the whole suite runs in seconds:
- A sample config is staged into a temp directory
- The CLI runs with
--testing, printing every command it would execute parseCliCommandsfolds line-continued output into one entry per logical commandcompareCliBaselinediffs the result against the committed baseline
Paths are normalized to /TEMP_DIR and /REPO_ROOT, so baselines are portable between local runs and CI.
The mechanism is used in two places:
| Where | Configs | Baseline | What it pins |
|---|---|---|---|
packages/cli/test/cli-commands.diff.test.ts |
packages/cli/sample_configs/sample-config-{usecase}.yaml |
packages/cli/test/__snapshots__/cli-commands-{usecase}.baseline.json |
CLI behavior per use case — one config per concern |
starter_kits/test/starter-kit.diff.test.ts |
each kit's mdaa.yaml |
starter_kits/test/{kit}/baselines/cli-commands.baseline.json |
The commands a real kit produces end to end |
Sample Configs
Unlike app modules, which vary a single schema, CLI sample configs are partitioned one file per concern. Each isolates one aspect of command assembly so a baseline diff points at a single cause:
| Config | Concern |
|---|---|
sample-config-hierarchy.yaml |
Global → domain → env → module resolution; which fields merge, accumulate, or replace |
sample-config-env-templates.yaml |
env_templates at global and domain scope; template module ordering and overrides |
sample-config-refs.yaml |
{{org}}, {{context:...}}, {{env_var:...}} resolution, and {{region}}/{{account}}/{{partition}} pass-through |
sample-config-shell-values.yaml |
Adversarial values (spaces, quotes, $, $(...), backticks, &&, ;) through every quote-only config sink |
sample-config-orchestration.yaml |
Staging, bootstrap toggles, stage grouping and module ordering |
sample-config-npm-version.yaml |
The npm install branch: module_path@version composition and --tag |
sample-config-terraform.yaml |
module_type: tf — Terraform action mapping and -var arguments |
Add a case to the existing config that owns the concern. Only add a new config when the concern is genuinely new, and add its name to SAMPLE_CASES in the diff test.
Choose baseline coverage for behavior that is emergent from composition — the assembled command text, ordering, and merge outcomes. Field-level validation, parsing, and error handling belong in unit tests; do not duplicate them here.
Configs may deliberately contain values that are invalid or unsafe in a real deployment — that is the point of sample-config-shell-values.yaml. They are test fixtures, not user-facing examples, so the sample-config standards for app modules (minimal/comprehensive, inline schema documentation) do not apply.
SSM references ({{ssm-org:...}}, {{resolve:ssm:...}}) must not appear: they resolve inside a construct scope during synth and throw when the CLI resolves a config.
Baseline Files
Baselines record current behavior, including known defects. A defect is documented in the header comment of the config that exercises it, so a reader is not misled into treating the output as correct.
Never hand-edit a baseline. Change the CLI and regenerate, so the diff is the evidence of the change:
# From packages/cli
npm run test:update-baselines
# Compare a different CLI build against the committed baselines
MDAA_CLI_ENTRYPOINT_OVERRIDE=/path/to/packages/cli/lib/mdaa.js npx jest --testPathPattern=cli-commands
Because commands are inert text under --testing, an unreviewed baseline can hide a defect that a template diff would never surface. Read every changed line: a quoting change that looks cosmetic can alter what the shell delivers to the app.
File Naming
test/cli-commands.diff.test.tsfor the CLI's own command baselinestest/__snapshots__/cli-commands-{usecase}.baseline.jsonfor committed command baselinessample_configs/sample-config-{usecase}.yamlfor the configs that drive them
Coverage
The CLI requires 80% statement and 75% branch coverage (packages/cli/jest.config.js).
Starter Kit Test Selection
Starter kit tests are selected per kit by scripts/test/test_starter_kit.py, which runs only what a change affects. A CLI change selects every kit's CLI command baseline (detection path 5, matched on @aws-mdaa/cli in the nx affected set) but no module synths — command-string changes cannot alter template output.
This gate is required for correctness, not just speed. The CLI is never a kit module_path, and a CLI change touches no kit mdaa.yaml, so without it the module synth baselines still run and pass while the CLI baselines silently go stale.
When a CLI change intentionally alters command format, regenerate the kit baselines too:
npm run test:starter-kits # verify: affected kits only
UPDATE_BASELINES=true npm run test:starter-kits:all # regenerate all kits
Testing Python Code
MDAA includes Python testing for Lambda functions, Glue jobs, and other Python components.
Python Test Structure
package-name/
├── python-tests/ # Python testing directory
│ ├── pyproject.toml # Modern Python project config
│ ├── conftest.py # Shared test fixtures
│ ├── .venv/ # Virtual environment (auto-managed by uv)
│ └── test_*.py # Test files
├── src/ or lib/ # Python source code being tested
└── package.json # npm scripts including test:python
Running Python Tests
Prerequisites: Install uv
# From the repo root
npm run test:python # Affected packages only
npm run test:python:all # All packages
# From any package with python-tests/
npm run test:python # Runs via scripts/test/test_python_package.sh
# Direct uv commands (from a python-tests/ directory)
uv run pytest # Run all tests
uv run pytest --cov # Run with coverage
uv run pytest -v # Verbose output
Adding Python Tests to New Packages
- Create a
python-tests/directory withpyproject.toml,conftest.py, andtest_*.pyfiles - Add
"test:python": "bash ../../../../scripts/test/test_python_package.sh"to the package'spackage.jsonscripts (adjust the relative path depth as needed) - The centralized runner (
scripts/test/test_python_package.sh) no-ops gracefully ifpython-tests/doesn't exist
Testing Apps Locally
MDAA Apps can be tested like any CDK app using cdk synth/diff/deploy from the app directory, providing the necessary context values:
cdk synth --require-approval never \
-c org="<org>" \
-c env="<env>" \
-c domain="<domain>" \
-c module_configs="<path/to/config/file>" \
-c module_name="<module_name>" \
--all
Any changes to underlying dependencies (stacks, constructs) require rebuilding those packages first (npm run build:all from the repo root, or npm run build in each modified package).
CI Pipeline
The CI pipeline runs tests at multiple stages:
| Stage | Job | What Runs |
|---|---|---|
| prebuild | feature_merge_lint |
ESLint on affected packages |
| prebuild | feature_merge_lint_python |
Ruff on Python tools |
| prebuild | feature_validate_packages |
Package structure validation |
| build | feature_merge_build_test |
Build + unit tests + diff tests (incl. CLI command baselines) with coverage |
| test | feature_merge_python_test |
Python tests (reuses build cache) |
| test | feature_merge_starter_kit_generate |
Generates a child pipeline with one job per starter kit |
| test | sk_{kit} (child pipeline) |
Per-kit CLI command baseline + affected module synth baselines |
| test | feature_merge_test_docs |
Documentation build validation |
| analyze | feature_merge_sonarqube |
SonarQube analysis |
Each sk_{kit} job self-filters to what the change affects via scripts/test/test_starter_kit.py, so an unaffected kit runs nothing.
Adding Tests
New L2 Construct
- Create
test/{construct}.compliance.test.ts - Use
MdaaTestAppandTemplate.fromStack()for assertions - Call
testApp.checkCdkNagCompliance()to validate Nag rules - Assert on all compliance-related resource properties
- Ensure 80% branch and statement coverage
New L3 Construct
- Create
test/{construct}.compliance.test.tsfor compliance assertions - Create additional
test/{construct}.test.tsfiles for functional tests - Test constructor validation in
test/constructor-exceptions.test.ts - Ensure 80% branch and statement coverage
New App Module
- Create sample configs under
sample_configs/(minimal + comprehensive + variants) - Create
test/{module}.diff.test.tswith onebaselineDiffTestAppper sample config - Create
test/{module}.snapshot.test.tswith onesnapShotTestAppper sample config usingCreate.appProvider - Run
UPDATE_BASELINES=true npm testto generate initial baselines - Commit the baseline JSON files
- Ensure 80% statement coverage
New CLI Behaviour
- Add the case to the
sample_configs/sample-config-{usecase}.yamlthat owns the concern, or create a new config and register it inSAMPLE_CASES - Cover only composition-emergent behavior here; put field validation and parsing in unit tests
- Run
npm run test:update-baselinesfrompackages/cliand read every changed command line - Commit the baseline alongside the code change
Updating Infrastructure
When a construct change intentionally modifies CloudFormation output:
- Run
npm testand review the diff failures - Confirm the changes are expected
- Run
npm run test:update-baselinesto accept the new baselines - Commit the updated baseline files alongside the code change
Updating Command Baselines
When a CLI change intentionally modifies emitted commands:
- Run
npm testinpackages/cliand review the failures - Regenerate with
npm run test:update-baselinesfrompackages/cli - Regenerate the kit baselines with
UPDATE_BASELINES=true npm run test:starter-kits:all - Verify each diff hunk is explained by the change; a semantically inert diff (requoting, path normalization) still needs to be attributed
- Commit both sets of baselines with the code change