Sidecars
Sidecars are additional containers that run along side the main container. They are usually used to perform peripheral tasks such as logging, configuration, or proxying requests.
Attention
Sidecars are not supported for Request-Driven Web Services.
Attention
If your main container is using a Windows image, FireLens, AWS X-Ray, and AWS App Mesh are not supported. Please check if your sidecar container supports Windows.
AWS also provides some plugin options that can be seamlessly incorporated with your ECS service, including but not limited to FireLens, AWS X-Ray, and AWS App Mesh.
If you have defined an EFS volume for your main container through the storage field in the manifest, you can also mount that volume in any sidecar containers you have defined.
How to add sidecars with Copilot?
There are two ways of adding sidecars using the Copilot manifest: by specifying general sidecars or by using sidecar patterns.
General sidecars
You'll need to provide the URL for the sidecar image. Optionally, you can specify the port you'd like to expose and the credential parameter for private registry.
port Integer
Port of the container to expose (optional).
image String or Map
Image URL for the sidecar container (required).
image.build String or Map
Build a container from a Dockerfile with optional arguments. Mutually exclusive with image.location.
If you specify a string, Copilot interprets it as the path to your Dockerfile. It will assume that the dirname of the string you specify should be the build context. The manifest:
image:
build: path/to/dockerfile
$ docker build --file path/to/dockerfile path/to
You can also specify build as a map:
image:
build:
dockerfile: path/to/dockerfile
context: context/dir
target: build-stage
cache_from:
- image:tag
args:
key: value
$ docker build --file path/to/dockerfile --target build-stage --cache-from image:tag --build-arg key=value context/dir.
You can omit fields and Copilot will do its best to understand what you mean. For example, if you specify context but not dockerfile, Copilot will run Docker in the context directory and assume that your Dockerfile is named "Dockerfile." If you specify dockerfile but no context, Copilot assumes you want to run Docker in the directory that contains dockerfile.
All paths are relative to your workspace root.
image.location String
Instead of building a container from a Dockerfile, you can specify an existing image name. Mutually exclusive with image.build.
The location field follows the same definition as the image parameter in the Amazon ECS task definition.
Warning
If you are passing in a Windows image, you must add platform: windows/x86_64 to your manifest.
If you are passing in an ARM architecture-based image, you must add platform: linux/arm64 to your manifest.
image.credentials String
An optional credentials ARN for a private repository. The credentials field follows the same definition as the credentialsParameter in the Amazon ECS task definition.
image.labels Map
An optional key/value map of Docker labels to add to the container.
image.depends_on Map
An optional key/value map of Container Dependencies to add to the container. The key of the map is a container name and the value is the condition to depend on. Valid conditions are: start, healthy, complete, and success. You cannot specify a complete or success dependency on an essential container.
For example:
image:
build: ./Dockerfile
depends_on:
nginx: start
startup: success
nginx sidecar has started and the startup container has completed successfully.
essential Bool
Whether the sidecar container is an essential container (optional, default true).
credentialsParameter String
ARN of the secret containing the private repository credentials (optional).
variables Map
Environment variables for the sidecar container (optional)
secrets Map
Secrets to expose to the sidecar container (optional)
env_file String
The path to a file from the root of your workspace containing the environment variables to pass to the sidecar container. For more information about the environment variable file, see Considerations for specifying environment variable files.
mount_points Array of Maps
Mount paths for EFS volumes specified at the service level (optional).
mount_points.source_volume String
Source volume to mount in this sidecar (required).
mount_points.path String
The path inside the sidecar container at which to mount the volume (required).
mount_points.read_only Boolean
Whether to allow the sidecar read-only access to the volume (default true).
labels Map
Docker labels to apply to this container (optional).
depends_on Map
Container dependencies to apply to this container (optional).
entrypoint String or Array of Strings
Override the default entrypoint in the sidecar.
# String version.
entrypoint: "/bin/entrypoint --p1 --p2"
# Alteratively, as an array of strings.
entrypoint: ["/bin/entrypoint", "--p1", "--p2"]
command String or Array of Strings
Override the default command in the sidecar.
# String version.
command: ps au
# Alteratively, as an array of strings.
command: ["ps", "au"]
healthcheck Map
Optional configuration for sidecar container health checks.
healthcheck.command Array of Strings
The command to run to determine if the sidecar container is healthy.
The string array can start with CMD to execute the command arguments directly, or CMD-SHELL to run the command with the container's default shell.
healthcheck.interval Duration
Time period between health checks, in seconds. Default is 10s.
healthcheck.retries Integer
Number of times to retry before container is deemed unhealthy. Default is 2.
healthcheck.timeout Duration
How long to wait before considering the health check failed, in seconds. Default is 5s.
healthcheck.start_period Duration
Length of grace period for containers to bootstrap before failed health checks count towards the maximum number of retries. Default is 0s.
Example
Sidecars with environment overrides
Similar to other service/job manifest fields, sidecars configurations can be overridden per-environment via the environments field.
Below is an example that configures the value for the DD_APM_ENABLED environment variable of the datadog sidecar, based on whether it is dev environment:
name: api
type: Load Balanced Web Service
sidecars:
datadog:
port: 80
image:
build: src/reverseproxy/Dockerfile
variables:
DD_APM_ENABLED: true
environments:
dev:
sidecars:
datadog:
variables:
DD_APM_ENABLED: false
nginx sidecar container
Below is an example of specifying the nginx sidecar container in a load balanced web service manifest.
name: api
type: Load Balanced Web Service
image:
build: api/Dockerfile
port: 3000
http:
path: 'api'
healthcheck: '/api/health-check'
# Target container for Load Balancer is our sidecar 'nginx', instead of the service container.
target_container: 'nginx'
cpu: 256
memory: 512
count: 1
sidecars:
nginx:
port: 80
image:
build: src/reverseproxy/Dockerfile
variables:
NGINX_PORT: 80
EFS volume in both the service and sidecar container
storage:
volumes:
myEFSVolume:
path: '/etc/mount1'
read_only: false
efs:
id: fs-1234567
sidecars:
nginx:
port: 80
image: 1234567890.dkr.ecr.us-west-2.amazonaws.com/reverse-proxy:revision_1
variables:
NGINX_PORT: 80
mount_points:
- source_volume: myEFSVolume
path: '/etc/mount1'
AWS Distro for OpenTelemetry sidecar
Below is an example of running the AWS Distro for OpenTelemetry sidecar with a custom configuration. The example custom configuration will not only collect X-Ray trace data, but also ship ECS metrics to a third party. The example will require an SSM secret and additional IAM permissions.
To use the OpenTelemetry sidecar, first, create a valid configuration file. Next, check the size of the configuration file. A standard parameter is limited to 4KB. If the configuration file is larger than 4K, an advanced SSM parameter must be used.
If an advanced parameter is required, it will need to be created and tagged manually. If the configuration fits within a standard parameter, create an SSM secret using the secret init. The YAML document below can be used as-is with New Relic after updating the API key written as "YOUR-API-KEY-HERE".
In the example YAML, the inclusion of empty keys is deliberate. The sidecar will use the collector defaults for those keys.
receivers:
awsxray:
transport: udp
awsecscontainermetrics:
processors:
batch:
exporters:
awsxray:
region: us-west-2
otlp:
endpoint: otlp.nr-data.net:4317
headers:
api-key: YOUR-API-KEY-HERE
service:
pipelines:
traces:
receivers: [awsxray]
processors: [batch]
exporters: [awsxray]
metrics:
receivers: [awsecscontainermetrics]
exporters: [otlp]
Writing X-Ray traces needs additional IAM permissions as shown below. Include this in addons according to the published documentation
Resources:
XrayWritePolicy:
Type: AWS::IAM::ManagedPolicy
Properties:
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: CopyOfAWSXRayDaemonWriteAccess
Effect: Allow
Action:
- xray:PutTraceSegments
- xray:PutTelemetryRecords
- xray:GetSamplingRules
- xray:GetSamplingTargets
- xray:GetSamplingStatisticSummaries
Resource: "*"
Outputs:
XrayAccessPolicyArn:
Description: "The ARN of the ManagedPolicy to attach to the task role."
Value: !Ref XrayWritePolicy
The configuration for the OpenTelemetry collector will be passed into the sidecar as an environment variable.
sidecars:
otel_sidecar:
image: 'public.ecr.aws/aws-observability/aws-otel-collector:latest'
secrets:
AOT_CONFIG_CONTENT: /copilot/${COPILOT_APPLICATION_NAME}/${COPILOT_ENVIRONMENT_NAME}/secrets/otel_config
Sidecar patterns
Sidecar patterns are predefined Copilot sidecar configurations. For now, the only supported pattern is FireLens, but we'll add more in the future!
# In the manifest.
logging:
# The Fluent Bit image. (Optional, we'll use "public.ecr.aws/aws-observability/aws-for-fluent-bit:stable" by default)
image: <image URL>
# The configuration options to send to the FireLens log driver. (Optional)
destination:
<config key>: <config value>
# Whether to include ECS metadata in logs. (Optional, default to true)
enableMetadata: <true|false>
# Secret to pass to the log configuration. (Optional)
secretOptions:
<key>: <value>
# The full config file path in your custom Fluent Bit image. (Optional)
configFilePath: <config file path>
# Environment variables for the sidecar container. (Optional)
variables:
<key>: <value>
# Secrets to expose to the sidecar container. (Optional)
secrets:
<key>: <value>
logging:
destination:
Name: cloudwatch
region: us-west-2
log_group_name: /copilot/sidecar-test-hello
log_stream_prefix: copilot/
You might need to add necessary permissions to the task role so that FireLens can forward your data. You can add permissions by specifying them in your addons. For example:
Resources:
FireLensPolicy:
Type: AWS::IAM::ManagedPolicy
Properties:
PolicyDocument:
Version: '2012-10-17'
Statement:
- Effect: Allow
Action:
- logs:CreateLogStream
- logs:CreateLogGroup
- logs:DescribeLogStreams
- logs:PutLogEvents
Resource: "<resource ARN>"
Outputs:
FireLensPolicyArn:
Description: An addon ManagedPolicy gets used by the ECS task role
Value: !Ref FireLensPolicy
Info
Since the FireLens log driver can route your main container's logs to various destinations, the svc logs command can track them only when they are sent to the log group we create for your Copilot service in CloudWatch.