MDAA TS Docs
    Preparing search index...

    Interface CrawlerDefinition

    Configuration for a Glue crawler including execution role, targets, scheduling, and schema management.

    Use cases: Automated S3 data discovery, scheduled database cataloging, schema change detection, data lake metadata management

    AWS: AWS Glue crawler configuration for automated data discovery and Glue catalog population

    Validation: executionRoleArn, databaseName, description, and targets are required; schedule, schemaChangePolicy, and tablePrefix are optional

    interface CrawlerDefinition {
        classifiers?: string[];
        databaseName: string;
        description: string;
        executionRoleArn: string;
        extraConfiguration?: ConfigurationElement;
        recrawlBehavior?: string;
        schedule?: ScheduleProperty;
        schemaChangePolicy?: SchemaChangePolicyProperty;
        tablePrefix?: string;
        targets: CrawlerTargets;
    }
    Index

    Properties

    classifiers? databaseName description executionRoleArn extraConfiguration? recrawlBehavior? schedule? schemaChangePolicy? tablePrefix? targets

    Properties

    classifiers?: string[]

    Name of the custom classifier to use from the crawler.yaml configuration

    databaseName: string

    Name of the database to crawl from the crawler.yaml configuration.

    description: string

    Description for the Crawler

    executionRoleArn: string

    Arn of the execution role

    extraConfiguration?: ConfigurationElement

    Crawler configuration as a string. See: https://docs.aws.amazon.com/glue/latest/dg/crawler-configuration.html

    recrawlBehavior?: string

    Recrawl behaviour: CRAWL_NEW_FOLDERS_ONLY or CRAWL_EVERYTHING or CRAWL_EVENT_MODE

    schedule?: ScheduleProperty

    Cron or rate schedule for automated periodic crawler execution.

    Use cases: Automated data discovery, scheduled catalog updates, periodic schema detection

    AWS: AWS Glue crawler schedule configuration

    Validation: Must be valid CfnCrawler.ScheduleProperty if provided

    schemaChangePolicy?: SchemaChangePolicyProperty

    Policy controlling how the crawler handles detected schema modifications.

    Use cases: Schema evolution management, table structure change handling, metadata consistency

    AWS: AWS Glue crawler schema change policy

    Validation: Must be valid CfnCrawler.SchemaChangePolicyProperty if provided

    tablePrefix?: string

    String prefix prepended to all table names created by the crawler.

    Use cases: Table naming organization, namespace management, naming conflict avoidance

    AWS: AWS Glue crawler table prefix

    Validation: Must be a valid string if provided

    targets: CrawlerTargets

    Crawler targets specifying data sources (S3, JDBC, DynamoDB, catalog) to crawl.

    Use cases: Data source specification, multi-source crawling, comprehensive data discovery

    AWS: AWS Glue crawler targets configuration

    Validation: Must be a valid CrawlerTargets object; required

    Settings

    Member Visibility

    On This Page

    Properties