Scheduled Job

List of all available properties for a 'Scheduled Job' manifest. To learn about Copilot jobs, see the Jobs concept page.

Sample scheduled job manifest

    name: report-generator
    type: Scheduled Job

    on:
      schedule: "@daily"
    cpu: 256
    memory: 512
    retries: 3
    timeout: 1h

    image:
      build: ./Dockerfile

    variables:
      LOG_LEVEL: info
    env_file: log.env
    secrets:
      GITHUB_TOKEN: GITHUB_TOKEN

    # You can override any of the values defined above by environment.
    environments:
      prod:
        cpu: 2048
        memory: 4096

name String
The name of your job.

type String
The architecture type for your job. Currently, Copilot only supports the "Scheduled Job" type for tasks that are triggered either on a fixed schedule or periodically.

on Map
The configuration for the event that triggers your job.

on.schedule String
You can specify a rate to periodically trigger your job. Supported rates:

Rate	Identical to	In human-readable text and `UTC`, it runs ...
`"@yearly"`	`"cron(0 * * * ? *)"`	at midnight on January 1st
`"@monthly"`	`"cron(0 0 1 * ? *)"`	at midnight on the first day of the month
`"@weekly"`	`"cron(0 0 ? * 1 *)"`	at midnight on Sunday
`"@daily"`	`"cron(0 0 * * ? *)"`	at midnight
`"@hourly"`	`"cron(0 * * * ? *)"`	at minute 0

"@every {duration}" (For example, "1m", "5m")
"rate({duration})" based on CloudWatch's rate expressions

Alternatively, you can specify a cron schedule if you'd like to trigger the job at a specific time:

"* * * * *" based on the standard cron format.
"cron({fields})" based on CloudWatch's cron expressions with six fields.

Finally, you can disable the job from triggering by setting the schedule field to none:

on:
  schedule: "none"

image Map
The image section contains parameters relating to the Docker build configuration or referring to an existing container image.

image.build String or Map
Build a container from a Dockerfile with optional arguments. Mutually exclusive with image.location.

If you specify a string, Copilot interprets it as the path to your Dockerfile. It will assume that the dirname of the string you specify should be the build context. The manifest:

image:
  build: path/to/dockerfile

will result in the following call to docker build: $ docker build --file path/to/dockerfile path/to

You can also specify build as a map:

image:
  build:
    dockerfile: path/to/dockerfile
    context: context/dir
    target: build-stage
    cache_from:
      - image:tag
    args:
      key: value

In this case, Copilot will use the context directory you specified and convert the key-value pairs under args to --build-arg overrides. The equivalent docker build call will be: $ docker build --file path/to/dockerfile --target build-stage --cache-from image:tag --build-arg key=value context/dir.

You can omit fields and Copilot will do its best to understand what you mean. For example, if you specify context but not dockerfile, Copilot will run Docker in the context directory and assume that your Dockerfile is named "Dockerfile." If you specify dockerfile but no context, Copilot assumes you want to run Docker in the directory that contains dockerfile.

All paths are relative to your workspace root.

image.location String
Instead of building a container from a Dockerfile, you can specify an existing image name. Mutually exclusive with image.build. The location field follows the same definition as the image parameter in the Amazon ECS task definition.

Warning

If you are passing in a Windows image, you must add platform: windows/x86_64 to your manifest.
If you are passing in an ARM architecture-based image, you must add platform: linux/arm64 to your manifest.

image.credentials String
An optional credentials ARN for a private repository. The credentials field follows the same definition as the credentialsParameter in the Amazon ECS task definition.

image.labels Map
An optional key/value map of Docker labels to add to the container.

image.depends_on Map
An optional key/value map of Container Dependencies to add to the container. The key of the map is a container name and the value is the condition to depend on. Valid conditions are: start, healthy, complete, and success. You cannot specify a complete or success dependency on an essential container.

For example:

image:
  build: ./Dockerfile
  depends_on:
    nginx: start
    startup: success

In the above example, the task's main container will only start after the nginx sidecar has started and the startup container has completed successfully.

entrypoint String or Array of Strings
Override the default entrypoint in the image.

# String version.
entrypoint: "/bin/entrypoint --p1 --p2"
# Alteratively, as an array of strings.
entrypoint: ["/bin/entrypoint", "--p1", "--p2"]

command String or Array of Strings
Override the default command in the image.

# String version.
command: ps au
# Alteratively, as an array of strings.
command: ["ps", "au"]

cpu Integer
Number of CPU units for the task. See the Amazon ECS docs for valid CPU values.

memory Integer
Amount of memory in MiB used by the task. See the Amazon ECS docs for valid memory values.

platform String
Operating system and architecture (formatted as [os]/[arch]) to pass with docker build --platform. For example, linux/arm64 or windows/x86_64. The default is linux/x86_64.

Override the generated string to build with a different valid osfamily or architecture. For example, Windows users might change the string

platform: windows/x86_64

which defaults to WINDOWS_SERVER_2019_CORE, using a map:

platform:
  osfamily: windows_server_2019_full
  architecture: x86_64

platform:
  osfamily: windows_server_2019_core
  architecture: x86_64

platform:
  osfamily: windows_server_2022_core
  architecture: x86_64

platform:
  osfamily: windows_server_2022_full
  architecture: x86_64

retries Integer
The number of times to retry the job before failing.

timeout Duration
How long the job should run before it aborts and fails. You can use the units: h, m, or s.

network.vpc Map
Subnets and security groups attached to your tasks.

network.vpc.placement String or Map
When using it as a string, the value must be one of 'public' or 'private'. Defaults to launching your tasks in public subnets.

Info

If you launch tasks in 'private' subnets and use a Copilot-generated VPC, Copilot will automatically add NAT Gateways to your environment for internet connectivity. (See pricing.) Alternatively, when running copilot env init, you can import an existing VPC with NAT Gateways, or one with VPC endpoints for isolated workloads. See our custom environment resources page for more.

When using it as a map, you can specify in which subnets Copilot should launch ECS tasks. For example:

network:
  vpc:
    placement:
      subnets: ["SubnetID1", "SubnetID2"]

network.vpc.placement.subnets Array of Strings or Map
As a list of strings, the subnet IDs where Copilot should launch ECS tasks.

As a map, the name-value pairs by which to filter your subnets. Note that the filters are joined with an AND, and the values for each filter are joined by an OR. For example, both subnets with tag set org: bi and type: public, and subnets with tag set org: bi and type: private will be matched by

network:
  vpc:
    placement:
      subnets:
        from_tags:
          org: bi
          type:
            - public
            - private

network.vpc.placement.subnetsfrom_tags Map of String and String or Array of Strings
Tag sets by which to filter subnets where Copilot should launch ECS tasks.

network.vpc.security_groups Array of Strings or Map
Additional security group IDs associated with your tasks.

network:
  vpc:
    security_groups: [sg-0001, sg-0002]

Copilot includes a security group so containers within your environment can communicate with each other. To disable the default security group, you can specify the Map form:

network:
  vpc:
    security_groups:
      deny_default: true
      groups: [sg-0001, sg-0002]

network.vpc.security_groups.from_cfn String
The name of a CloudFormation stack export.

network.vpc.security_groups.deny_default Boolean
Disable the default security group that allows ingress from all services in your environment.

network.vpc.security_groups.groups Array of Strings
Additional security group IDs associated with your tasks.

network.vpc.security_groups.groupsfrom_cfn String
The name of a CloudFormation stack export.

variables Map
Key-value pairs that represent environment variables that will be passed to your job. Copilot will include a number of environment variables by default for you.

secrets Map
Key-value pairs that represent secret values from AWS Systems Manager Parameter Store that will be securely passed to your job as environment variables.

storage Map
The Storage section lets you specify external EFS volumes for your containers and sidecars to mount. This allows you to access persistent storage across regions for data processing or CMS workloads. For more detail, see the storage page.

storage.volumes Map
Specify the name and configuration of any EFS volumes you would like to attach. The volumes field is specified as a map of the form:

volumes:
  <volume name>:
    path: "/etc/mountpath"
    efs:
      ...

storage.volumes.<volume> Map
Specify the configuration of a volume.

storage.volumes.<volume>.path String
Required. Specify the location in the container where you would like your volume to be mounted. Must be fewer than 242 characters and must consist only of the characters a-zA-Z0-9.-_/.

storage.volumes.<volume>.read_only Boolean
Optional. Defaults to true. Defines whether the volume is read-only or not. If false, the container is granted elasticfilesystem:ClientWrite permissions to the filesystem and the volume is writable.

storage.volumes.<volume>.efs Map
Specify more detailed EFS configuration.

storage.volumes.<volume>.efs.id String
Required. The ID of the filesystem you would like to mount.

storage.volumes.<volume>.efs.root_dir String
Optional. Defaults to /. Specify the location in the EFS filesystem you would like to use as the root of your volume. Must be fewer than 255 characters and must consist only of the characters a-zA-Z0-9.-_/. If using an access point, root_dir must be either empty or / and auth.iam must be true.

storage.volumes.<volume>.efs.auth Map
Specify advanced authorization configuration for EFS.

storage.volumes.<volume>.efs.auth.iam Boolean
Optional. Defaults to true. Whether or not to use IAM authorization to determine whether the volume is allowed to connect to EFS.

storage.volumes.<volume>.efs.auth.access_point_id String
Optional. Defaults to "". The ID of the EFS access point to connect to. If using an access point, root_dir must be either empty or / and auth.iam must be true.

logging Map
The logging section contains log configuration parameters for your container's FireLens log driver (see examples here).

logging.image Map
Optional. The Fluent Bit image to use. Defaults to public.ecr.aws/aws-observability/aws-for-fluent-bit:stable.

logging.destination Map
Optional. The configuration options to send to the FireLens log driver.

logging.enableMetadata Map
Optional. Whether to include ECS metadata in logs. Defaults to true.

logging.secretOptions Map
Optional. The secrets to pass to the log configuration.

logging.configFilePath Map
Optional. The full config file path in your custom Fluent Bit image.

publish Map
The publish section allows services to publish messages to one or more SNS topics.

publish:
  topics:
    - name: orderEvents

In the example above, this manifest declares an SNS topic named orderEvents that other worker services deployed to the Copilot environment can subscribe to. An environment variable named COPILOT_SNS_TOPIC_ARNS is injected into your workload as a JSON string.

In JavaScript, you could write:

const {orderEvents} = JSON.parse(process.env.COPILOT_SNS_TOPIC_ARNS)

For more details, see the pub/sub page.

publish.topics Array of topics
List of topic objects.

publish.topics.topic Map
Holds configuration for a single SNS topic.

publish.topics.topic.name String
Required. The name of the SNS topic. Must contain only upper and lowercase letters, numbers, hyphens, and underscores.

publish.topics.topic.fifo Boolean or Map
FIFO (first in, first out) SNS topic configuration.
If you specify true, Copilot will create the topic with FIFO ordering.

publish:
  topics:
    - name: mytopic
      fifo: true

Alternatively, you can also configure advanced SNS FIFO topic settings.

publish:
  topics:
    - name: mytopic
      fifo:
        content_based_deduplication: true

publish.topics.topic.fifo.content_based_deduplication Boolean
If the message body is guaranteed to be unique for each published message, you can enable content-based deduplication for the SNS FIFO topic.

environments Map
The environment section lets you override any value in your manifest based on the environment you're in. In the example manifest above, we're overriding the CPU parameter so that our production container is more performant.